noironetworks / opflex
79%
master: 68%

LAST BUILD BRANCH: drop-log-redirect-args
DEFAULT BRANCH: master
Repo Added
Files 285
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH jefferson-svc-stats-thread
branch: jefferson-svc-stats-thread
CHANGE BRANCH
x

1644

push

travis-ci-com

Gautam Venkataramanan 
Changes to move svc stats to a low prio thread

- Created a new thread to handle svc stat flows CRUD. The thread works on a new asio io_service object to handle new events from taskqueue.
- Stats collection done via ServiceStatsManager also uses the new asio io_service.
- All linux threads by default have SCHED_OTHER scheduling policy. This has a static priority of 0. Other sched policies like SCHED_FIFO and SCHED_RR allow users to control thread priority varying from values 1 to 99. Due to this FIFO/RR will preempt SCHED_OTHER threads. Either we move all threads to FIFO/RR and set priorities or somehow influence priority within SCHED_OTHER policy. Luckily there is a "nice" value than is configurable at a thread level per process which can control the total time allocated per thread. The new thread has highest "nice" value, which makes it consume least CPU time.
- Call programServiceSnatDnatFlows() appropriately per service, once all the stats flows are programmed. This is to update cookie values in SNAT/DNAT flows for stats collection. Note: This used to be called twice for every service during EP update. Reduced this to just one call. Also, since service flows get programmed via agent thread and since stats flows are created via different thread, this call needs to be called separately for service updates as well.
- Flows get programmed by both agent thread and new thread. Addressed tsan issues with a mutex in switch manager.
- Changed a few booleans to atomic bools.
- enabled svc stats by default
- make check and mock agent work fine.
- reproduced the issue faced by telenor on fab1 with old opflex-agent image. 177 services + 110 pods per node took 2 hours to get applied. 40% CPU consumed by opflex-agent consistently. A new service-pod when spawned, took around 2 mins 20 seconds to be responsive. With the fix, CPU consumption during flow creation is around 25%. New service-pod is accessible instantaneously. Bootup with new image was also v quick.

S... (continued)

22170 of 27941 relevant lines covered (79.35%)

3290.11 hits per line

Relevant lines Covered
27941 RELEVANT LINES 22170 COVERED LINES
3290.11 HITS PER LINE
Source Files on jefferson-svc-stats-thread

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
1644 jefferson-svc-stats-thread Changes to move svc stats to a low prio thread - Created a new thread to handle svc stat flows CRUD. The thread works on a new asio io_service object to handle new events from taskqueue. - Stats collection done via ServiceStatsManager also uses t... push Gautam Venkataramanan travis-ci-com pending completion  
See All Builds (2774)