The Micro-burst that Killed my Network


RSS Feed Subscribe to EtherNEWS Bookmark and Share

The IT department at the stock trading firm was under serious pressure.  Trading times, carefully monitored by sophisticated monitoring systems, were becoming increasingly irregular – latency was creeping into the tens of milliseconds, twice as long as previous weeks, and certainly slower than competitors’ systems.  Even without a glance at trading times the brokers knew something was amiss – revenue was dropping as they suspected their trades were being executed just behind others beating them to the markets.

But things didn’t add up – outside their company walls they were running at 50% of their available access link bandwidth, and network latency measurements showed far less than what the trade monitoring system reported.  The same was true in the LAN and in the data center – each appeared to be working at peak performance.  There was something they didn’t see, and it was costing them millions.

Today almost half of all trades executed globally are initiated and completed by computers, not humans.  These algorithmic trading platforms, as they are known, constantly scan international markets for price discrepancies that offer a nearly instantaneous, guaranteed return to financial institutions that can move near the speed of light to buy and sell across global markets.  It’s a proven and increasingly important strategy, but you have to be fast.

Although stock trading is an industry that is severely affected by network performance issues, many other verticals and are similarly affected – anything transactional that involves time or money feels the impact of delays, packet loss and capacity issues.  Today we’re not just running networks that have to keep up with the speed of business, they define the speed limit.

So what was happening back at the broker?  What were they missing?  Knowing nothing had changed inside their company walls, they turned to their service provider for help.

The Microburst Phenomenon

Their operator, specialized in serving financial markets, had seen this scenario before, and luckily their networks were well instrumented.  They had visibility on per-flow, one-way latency to microsecond resolution, but their measurements also turned up good results.  That’s when they knew it was time to look deeper.  By increasing the granularity of their utilization monitoring to a per-second basis, they discovered that even though bandwidth utilization appeared normal over their standard, five minute monitoring intervals, there were microbursts of data that went well beyond the commissioned bandwidth – up to 140% – if even momentarily.

 

Micro-burst Traffic Profile Monitoring

Micro-burst Traffic Profile Monitoring

 

They were measuring these traffic stats just before the trader’s data entered their network, before their network interface devices’ (NID’s) regulators, so they could see the micro-bursts that were resulting in very short-term packet loss – just a few frames dropped as the peaks hit.  This small, almost negligible loss meant trading time would nearly double – the missing packets needed to be retransmitted to complete the buy or sell request, adding precious milliseconds to the transaction.

How did they fix the problem?  There are a number of ways to react when you know micro-bursting is affecting application performance.  A simple solution would be to increase committed or excess bandwidth or burst size limits, another would be to shape and smooth out any traffic not sensitive to latency or jitter sharing the same link.  With advanced traffic monitoring, classification and per-flow conditioning at the ingress to the network, either the service provider or the end-user can optimize their service flows for bandwidth efficiency, performance, or a combination of both depending on the each application’s requirements.

Micro-bursts are not a new thing – they happen all the time, over all kinds of networks, and affect a wide range of applications.   We just don’t notice them because of their highly transient nature.  The evolution of Ethernet & IP / MPLS performance monitoring has reached a point where not only are measurements highly precise, they are also highly granular – we now have the tools we need to detect these short-term events and take action to ensure applications are running at peak performance.  With these capabilities now available in cost efficient NIDs and monitoring platforms as a standard feature – they are easily in reach of both service providers and their customers.  When performance puts your business ahead, you now have the ability to set your own speed limit and avoid those million dollar tickets.

RSS Feed Subscribe to EtherNEWS Bookmark and Share

43 Responses to “The Micro-burst that Killed my Network”

  1. Delta Houtz says:

    Interesting article. Were did you acquire all the information from :D

  2. Broker says:

    Very cool site and I just bookmarked ya too!

  3. Juanita Hardiman says:

    The big banks and Wall Street took billions in bailout money from taxpayers and then turned it into huge profit. An insider’s club report showed last week a simple step by step strategy to spying on the big bank’s trades so we can profit with them.

    Banks strategies — May be useful to know why we never get out from under our banker’s thumb..It is sure the big banks dont want us to see this.

  4. Great information! I

  5. The company brokerages network will provide you with access to some large pool of individuals who’ve the information about companies for sale and buyers or investors looking for any organization venture. By making great use from the information you’ve, you might be cutting a offer and make a handsome profit out from the transactions.

  6. Chris Bloom says:

    With gig networks, a lot can happen is a second. For microburst analysis, WildPackets Compass Network Forensics Dashboard offers an interactive milliseconds view. Not only do you get a 1000 points of data in the graph for each second, but you can easily and remove different nodes and protocols to the graph as well. Very innovative app for OmniPeek Enterprise.

    https://mypeek.wildpackets.com/view_submission.php?id=107

  7. stinsbalt says:

    Hi there guys.
    I just registered here….
    I really love the atmosphere here. .
    Can you to check my health article my web site

  8. On this day and age the place people will go long methods and endure much discomfort uggs just to become counted among the “trendy” and “fashionable”, the Ugg Start genuinely stands out like a Pillar in opposition to the storm. Initially worn by Australian sheepherders to help keep their feet heat, these boot styles have considering that 2003, turn into very well-liked in the United States. Such men and women as Pamela Anderson and Kate Hudson are actually known to put on them to maintain their feet warm.

    Many individuals would ugg boots size 4 regard these Boot styles as “unfashionable” or “uncool” simply because the boot footwear appear big and odd. Some have even called to get a boycott on sporting these boot footwear due to their distaste for its seem. Despite all of this, the Ugg boot styles has long been enormously successful and is probably right here to remain.

    Now, you may possibly ask, “why?” or “how?” does a boot that defies all style logic remain well-known? The reply lies in two simple motives. Initial, some folks need to stand out through the crowd green ugg boots and differ within the norm. The Ugg boot footwear will surely make you obvious in a fashion-conscious crowd. 2nd, and more importantly, could be the fact that these boot styles are indeed really relaxed to dress in. These boot footwear have already been well-known with swimmers snugg boots and surfers considering that the 1970s and are regularly put on by surfers in California. The style with the kick out with its sheepskin results in substantial thermostatic advantages. The shoe also boasts fleecy fibers in the interior to enable for air circulation in the course of the summer so your ft to not get overheated. With this mixture of warming and cooling, the Ugg shoe is the perfect kick out for all occasions. Add to this reality that the sheepskin materials utilized to produce this start is usually more snug than the standard leather used to create the frequent trunk, and you’ve got a heat and snug shoe for all seasons.

    Despite the comfort good reasons, lots of people will still not put on these shoes because of their look. But Ugg shoes now come in a very variety of types, sizes and colours and may typically be incorporated in any kind of apparel you on wearing. Usually considering that these boots are usually worn inside winter months for thermal insulation, it is possible to put on any sort of Ski don and even very long jeans. In reality any type of extended winter insulated ebay ugg boots
    pants will glimpse very normal and fit properly with a pair of matching-color Uggs. If you might be nonetheless concerned about what to wear with them, there are lots of places on-line including the Yahoo Solutions Forum that will aid you pick out the correct attire to have on along with your boot styles.

    So with these fashion concerns answered, there’s no cause why you must not join the rush and buy a pair australian ugg of Ugg boots for this winter.

  9. the Yahoo Solutions Forum that will

  10. :.. I am very thankful to this topic because it really gives useful information “`-

  11. Oh my goodness you have got probably the prettiest things I have discovered! I love your weblog!

  12. I like to visit your blog a couple times a week for new readings. I was wondering if you have any other subjects you write within (easy) reach? You’re a singularly interesting Colloq pen-pusher.

  13. Hello, i think that i saw you visited my weblog thus i came to “return the favor”.I’m trying to find things to improve my website!I suppose its ok to use a few of your ideas!! food stamps illinois

  14. ecnuzjfqec says:

    gabxwbddfejbo, kthqfiwnzs , [url=http://www.ibvpopecbp.com]tdqqqpfzdk[/url], http://www.ajdoofjieo.com kthqfiwnzs

  15. One thing I would really like to say is always that before getting more laptop or computer memory, consider the machine in to which it would be installed. Should the machine is definitely running Windows XP, for instance, the particular memory limit is 3.25GB. Applying greater than this would merely constitute any waste. Be sure that one’s mother board can handle the upgrade quantity, as well. Thanks for your blog post.

  16. plays says:

    Can’t see your pictures… what’s wrong?

  17. BG mail says:

    I must show appreciation to this writer for rescuing me from this type of challenge. After exploring throughout the search engines and getting suggestions which were not beneficial, I was thinking my life was done. Living minus the strategies to the problems you’ve fixed by means of your short post is a critical case, as well as ones that would have negatively damaged my career if I had not come across your blog post. Your own expertise and kindness in controlling all things was excellent. I am not sure what I would have done if I had not come upon such a subject like this. I am able to at this time relish my future. Thanks for your time so much for the high quality and sensible guide. I won’t hesitate to refer your blog post to anyone who wants and needs guide on this matter.

  18. winning says:

    Hello would you mind letting me know which hosting company you’re working with? I’ve loaded your blog in 3 different internet browsers and I must say this blog loads a lot faster then most. Can you recommend a good web hosting provider at a reasonable price? Thanks, I appreciate it!

  19. I thought your post was really good! I think you should post more ^^ Bookmarked this page!

  20. uggs outlets says:

    I really like what you guys are up too. This sort of clever work and exposure! Keep up the superb works guys I’ve you guys to my personal blogroll.

  21. We still cannot quite assume that I really could often be one of those particular staring at the important points seen on yuor web blog. My children and that i are sincerely thankful for your personal generosity as well as giving me possibility pursue our chosen profession path. Published info Managed to get through the web-site.

  22. effect says:

    Particularly helpful

  23. Starti–ng in the late 1970s, the U.S. government and corporatio–ns pushed to redefine “finance” from a service that supports the real economy to a tradable commodity whose flow across borders should be uninhibite–d. Starting in the late 1980s, they successful–ly pushed for financial services to be included in “trade” negotiatio–ns, including those establishi–ng the World Trade Organizati–on (WTO). “The sector was truly unique in that respect, and there is little doubt within the trade policy community that financial sector support in the European Union and the United States was a determinin–g force in concluding the FSA [WTO Financial Services Agreement]–” notes a study posted on the WTO’s own website “Financial Services and the WTO: What Next?”rs gold making guide,rs gold powerleveling,buy rs gold and items

  24. xnmzbbddfejbo, Attract Women, WmQxgBl, Finpecia, RPVEoRi, tetris, AnYVcDQ, Canadian viagra, ORhxaUM, swish templates, EojOfRO, how to attract a woman, bekQvFe.

  25. gfnnrbddfejbo, Order viagra cheap, ljziiAU, Generic cialis, OOxogZs, Buy Carisoprodol, rKFTybA, Igf-1, SCIxjxw, Buy Ativan, bVERXdk, Levitra free samples, YmrQKkg, Cialis, cJvdFPK, Generic Propecia, qMrYdaG.

  26. sspvxbddfejbo, Tinnitus Miracle, MexleIf, Priligy dapoxetine, buXAzMx, Intivar, dpySDZR, Side effects of ambien, ZaKuOpO, Phallosan, RKzMxqD, Hebrew baby names, mYMrGgO, cheapest car insurance, kjXEahM, Idol Lash, BZkNflP.

  27. Fake uggs on clearance are taller and don’t run a size bigger like real boots.

  28. what time are you all going to be there, in Gainesville. My son and I just came from regal and no one was there. He was greatly disappointed. He loved the movie though, but it would of been a nice touch for him to see you guys

  29. hey there and thank you in your info – I’ve certainly picked up something new from proper here. I did alternatively experience several technical points using this site, since I experienced to reload the website lots of times prior to I may get it to load properly. I had been wondering if your web hosting is OK? Now not that I am complaining, but slow loading cases instances will sometimes impact your placement in google and can harm your quality rating if ads and ***********|advertising|advertising|advertising and *********** with Adwords. Anyway I’m adding this RSS to my email and could look out for much more of your respective intriguing content. Ensure that you update this again soon..

  30. Hiba says:

    of course, i genuinely concur with you, i like your submit, i assure someday i’ll be back again to study you document all over again, actually enjoy it, i will bookmark it

  31. Hey there, I ‘m not use to comment but realized I would do it now. Plain and simple I like your writing.

  32. Does your site have a contact page? I’m having trouble locating it but, I’d like to shoot you an e-mail. I’ve got some ideas for your blog you might be interested in hearing. Either way, great site and I look forward to seeing it grow over time.

  33. Or semenax says:

    qvyxjbddfejbo, Generic Ativan, qnIilav, Maximum dose ativan, ktlRTBH, VigRX, PUgCQBu, Butalbital caffeine, NEtzMPr, Semenax, yCoWmDo, Buy Xanax, WzFPBHd.

  34. Priligy says:

    qaeyxbddfejbo, Priligy dapoxetine, RxvaPRl.

  35. Semenax says:

    axsnnbddfejbo, Buy Semenax, tdlbKef.

  36. Cory Ramirec says:

    How fun! I have been admiring those words on pinterest. Great idea to add them to the stairs.

Leave a Reply

You must be logged in to post an
interactive video comment.