from the Nippon-Nippon-Cha-Cha-Cha! dept.
Wired has a story about Twitter's ability to handle huge surges in traffic. From the article:
When the New Year arrives or even as they watch certain moments in shows and movies broadcast on national television, tens of thousands of Japanese will tweet at practically the same instant. “Everyone tweets at the New Year, but the Japanese are more in-sync,” says Hashemi, who, as Twitter’s director of site reliability engineering, works to make sure its mini-messaging service stays in good working order. “They do it at exactly midnight.”
This provides a small window into the unique culture of the Japanese, known for exhibiting a certain type of conformity, but there was a time when it was also an enormous problem for Twitter. As the year 2012 arrived in Japan, the country’s synchronized tweets crashed Twitter’s entire service, worldwide. It was 3pm in Britain when the site went belly-up.
The rest of the article delves into a very general overview of how Twitter engineered its way out of the problem.
(Score: 2) by WizardFusion on Monday September 08 2014, @01:29PM
Without reading the summary...
Yea, something gets fixed.
or
Oh no, it's all gone wrong.
:)
(Score: 1, Funny) by Anonymous Coward on Monday September 08 2014, @01:52PM
gullible lemmings...
(Score: 3, Interesting) by nyder on Monday September 08 2014, @01:54PM
Okay, I don't know what the big problem is.
First off, Japan has 1 timezone. So ya, everyone tweeting happy New Year shouldn't be very surprising when it happens. In fact, you know in advance when the new year starts, so maybe during that time you up the number of servers & connections you have?
You use past data on when you got the most tweets from any given area. If you seems like from 8-11pm you get lots more tweets in chunks (movies getting out), you ramp up the servers & connections during this time.
Isn't this what computers are for? To analyze the data so you can maximize your hardware when & where needed?
Sounds to me like Twitter is just cheap and not providing enough hardware to cover their busy times in Japan.
(Score: 5, Funny) by Snotnose on Monday September 08 2014, @02:08PM
There you go, thinking like an engineer again. How many times do I have to tell you Twitter is full of marketing types who think they just need to take it to the next level to turn it into a win-win situation. The best practice is to get rid of the low-hanging fruit first. Call a meeting and set an agenda so they can go flag up on this thing.
When the dust settled America realized it was saved by a porn star.
(Score: 1) by helel on Monday September 08 2014, @02:11PM
2012 is the past data. In 2013 they planned ahead and kept the servers up because they knew what they were facing as a result of that past data.
(Score: 4, Informative) by Lemming on Monday September 08 2014, @03:10PM
Just buying 10 times the hardware which will sit idle for most of the year isn't very cost effective. But much more important, simply scaling up the hardware will not solve the problem. Sending a tweet doesn't happen in a vacuum. If @alice sends a tweet, it should be visible (almost) immediately to her followers @bob and @charly. @bob and @charly are probably connected to other servers, possibly located on the other side of the globe.
So this is also a software problem, a network infrastructure problem,... On the side of software they talk about using Scala [wikipedia.org]. It's a language build for scalability, combining features like static typing and functional programming. This makes it very usable for e.g. massive parallel computing.
(Score: 2) by kaszz on Monday September 08 2014, @10:20PM
www.wired.com/2013/09/the-second-coming-of-java/2 [wired.com]:
Facebook is seemingly up the same path by moving its PHP code onto a custom-built virtual machine that provides just-in-time compilation.
So massive (ab)use of a service is only a problem if you rely on the cult of the super server..
I think the end conclusion is that interpretated bytecode has a performance penalty. but the benefit of consistent code API regardless of underlying platform outweighs this. The most important step according to the spirit of the article is to make a P2P network of independent processes that has a consistent API. So that any server or process that fails will be easy to replace. And recruitment is perhaps easier for javacoders than for C-coders.
(Score: 2) by halcyon1234 on Monday September 08 2014, @02:33PM
Gah, the wording makes this sound like one of those "OMG True Facts" urban legends that get passed around by email. Usually in the "lol other cultures are weird" section that present a "fact" without any consideration for reality. Like if you were to think about, it'd be an entire country of automaton-like, dead-eyed "foreign people" standing in a grid, cell phones in hand, inhuman faces lit in blue-- all getting the command to type and press sent EXACTLY AT MIDNIGHT.
Never mind reaction time of people. Clocks that aren't in synch by seconds, or minutes. Or someone taking a drink instead of pressing send, or hugging a loved one. Or drunk. Or typoing and having to correct their message. Nope, them silly foreigners ALL TYPE AT ONCE LOL!
Original Submission [thedailywtf.com]
(Score: 0) by Anonymous Coward on Monday September 08 2014, @08:42PM
Dude, you're talking about Japan. I would be surprised if they're not all running NTP.
Dude, you're still talking about Japan. It only takes two tentacles to hug a loved one, leaving plenty of tentacles to press "send".
Why do you assume they start typing at midnight instead of typing ahead of time and only having to press "send" at midnight? Plenty of time to correct typos.
s/TYPE/PRESSING SEND/
(Score: 2) by krishnoid on Monday September 08 2014, @11:00PM
Clocks that aren't in synch by seconds, or minutes.
The tweets are most likely coming from cell phones, whose clocks are all getting their time from NTT's single (?) network, so they're probably in synch by milliseconds.
Like if you were to think about, it'd be an entire country of automaton-like, dead-eyed "foreign people" standing in a grid, cell phones in hand, inhuman faces lit in blue-- all getting the command to type and press sent EXACTLY AT MIDNIGHT.
Or maybe all doing it because they want to feel united with many other people at once -- like everyone gathering or tuning in to watch the ball drop in NYC.
(Score: 2) by TheLink on Tuesday September 09 2014, @04:20AM
I'm sure their clocks are synced. Because it actually makes a difference. This should give you an idea of what things are like in Japan: http://www.nytimes.com/2005/04/27/world/asia/27iht-japan.html?_r=0 [nytimes.com]
In 2004, on the 40th anniversary of the bullet train, there was much hand-wringing over the fact that a year earlier the trains on that line had registered, on average, a delay - of six seconds.
https://en.wikipedia.org/wiki/Rail_transport_in_Japan#Punctuality [wikipedia.org]
Japanese railways are among the most punctual in the world. The average delay on the Tokaido Shinkansen in fiscal 2006 was only 0.3 minutes.[14] When trains are delayed for as little as five minutes, the conductor makes an announcement apologizing for the delay and the railway company may provide a "delay certificate" (遅延証明書), as no one would expect a train to be this late.
Basically a lot of people around the world do send messages when the New Year begins, but the Japanese "exact" is a lot more exact than it is in the rest of the world. Enough at least for Twitter to notice.
I wouldn't say their culture is weird, I'd say different from most of the world. Where else would staff apologize for a train's emergency stop due to an earthquake AND be told to do so in their training manual... http://www.kalzumeus.com/2011/03/13/some-perspective-on-the-japan-earthquake/ [kalzumeus.com]
One joke goes that during the World Cup, the Japanese football fans left the stadiums cleaner than they were before. ;)
(Score: 2, Funny) by Anonymous Coward on Monday September 08 2014, @03:48PM
That thinks it is hilarious that Japan, in unison, basically DDOS itself once a year?
America can't do anything this coordinated if we had a 100 tries.
(Score: 1, Funny) by Anonymous Coward on Monday September 08 2014, @05:25PM
False!!!!
There is worry every year during the Superbowl, for blowing sewer lines. It seams we all go to the bathroom during slow points in the game (normally commercials). This causes a sewer systems to get "flash floods". It is easy to add more "pipes" to a website... try that in the middle of your city.
The surge is used to tell if a commercial is "good" or "bad", less surge better the commercial.
But then again no: http://www.poconorecord.com/apps/pbcs.dll/article?AID=/20120205/NEWS/202050311/-1/SPORTS [poconorecord.com]
It is water usage!!! http://www.nbcnewyork.com/news/local/Super-Bowl-Causes-Spike-in-Water-Use-139009384.html [nbcnewyork.com]