SoylentNews is people
SoylentNews
The Census and Australian Bureau of Statistics websites have crashed on their most important day of the year [Census day for Australians], leaving Australians frustrated and unable to complete their form online. Many trying to log on to the Census and ABS sites were met with an error message on Tuesday evening saying the site could not be reached.
In a first, the census requires a full name to be filled in, though many including politicians are concerned about privacy issues. Crossbench senators including Nick Xenophon, Scott Ludlam and Sarah Hanson-Young are pledging not to comply with rules for the census will be treated like any other citizen and risk fines of $180 per day, the Australian Bureau of Statistics said on Tuesday.
Some took to Twitter to slam the ABS, with one user sarcastically saying they were "definitely technically competent enough to keep our most private data safe."
(Score: 5, Insightful) by ledow on Tuesday August 09 2016, @02:23PM
23 million people, all trying to fill in a Census in one day.
I mean, however you cut that, you're causing problems for yourselves there, surely? Can the organisation of that not be done better, i.e. sending out the questions early and "fill in the information as it will be / is / was on the Census Day". Why does the data have to be entered on the day it's collected, or aimed at?
But it's not beyond any big company (I believe Lockheed Martin did some of the UK census, for example) - Google Apps would probably laugh at that even if the census form was just a Google Apps Form plugging into Google Sheets (i.e. zero preparation required).
And HTTP errors, and the other stuff like blank pages and so on is just laughable. I mean, seriously, it's indicative that there is no content delivery network being used properly, no proper redundancy, and they are literally all just coming into the same database from everywhere. That's a disaster waiting to happen.
But even 23 million online simultaneous users is not beyond the realms of ordinary in the IT world, really. There must be 23 million people online on iPlayer over the course of a day, and video streaming is a damn sight heavier and more difficult to do en-masse.
All I can say is that their testing must have been abysmally contrived ("Oh, look, we can load the front page on 1,000 computers simultaneously without slowdown, it must be fine...").
(Score: 0) by Anonymous Coward on Tuesday August 09 2016, @02:43PM
Censuses have been done successfully for thousands of years, by distributing the work to an army of census workers who count their local neighborhoods. The Real World scales. It scales very very well.
Of course if you want a census done properly, you need to pay the army of census workers. And there lies the problem when you want to be fucking cheap and use one website instead of an army of many people.
Fuck IT.
(Score: 2) by c0lo on Tuesday August 09 2016, @03:19PM
I've got it on the vine: looks like it's expanding [wikipedia.org].
Perhaps the expansion stretched the network over the limit?
https://www.youtube.com/@ProfSteveKeen https://soylentnews.org/~MichaelDavidCrawford
(Score: 2, Interesting) by VLM on Tuesday August 09 2016, @02:50PM
Google Apps would probably laugh at that
I found a link claiming there's 50 million K12 students and I know the local school district deeply drinks the Kool Aid and uses google apps exclusively for everything, as in handouts, assignments, lecture slides/notes, quizzes, schedules, reports/papers/essays, everything. Even if you assume only 10% of the countries school districts drink the google kool aide, there's at least a 10:1 ratio of load between "who is ye" (or however you say in in Australian) vs K12 kids screwing around for an entire working school day. And at least in the USA those 50 million K12 students all are on vacation, so google should absolutely trivially be able to handle the load.
My guess is "noob disease" where some rocket surgeon decided to make the page completely dynamic and no caching so its gotta smack the database 50 times per pageload and obviously you can't use an "old fashioned" form you gotta AJAX all the way so every time a key is hit there's a round trip and at least one database hit.
Something I don't understand about Australia is their country is immense and their population is microscopic therefore their houses cost more than California prices because ... damnfino. Lack of trees to make wood to build? I donno. You'd think its the kind of country where $50 would get you enough acres to operate a cattle ranch, but no, a remodeled outhouse is like $1M.
(Score: 0) by Anonymous Coward on Tuesday August 09 2016, @03:11PM
Please tell us all again how little you know about Australia (most things?).
You can't even understand why people prefer to live in a city and not the middle of a desert...why are we not surprised.
(Score: 5, Interesting) by ledow on Tuesday August 09 2016, @03:55PM
I work in schools, which is exactly why I choose Google Apps as an example.
As I speak, several hundred children and dozens of staff are logged in and doing their homework and working on documents collaboratively (multiple cursors in the document and live-change propogation). Word processors, spreadsheets, forms, you name it.
The reason schools use it is thus:
1) It's free. Totally free, but Google Apps for Business level service. As many users as you like, as much storage as you like (literally Terabytes per user if we wanted), all the services you like, all free.
2) It integrates fabulously. Our webfilters, database software, library system, etc. can all query Google Apps for users and their categorisation and then we can do things like redirect their proxy on their Chromebooks accordingly. I can manage all the users from a smartphone app. There's a tie-in straight to our AD that either live-authenticates (SAML? Can't remember the acronym), or allows you to manually push sanitised user lists to the cloud (so Google know I have a user Fred Bloggs but have no way to log into our school systems as Fred Bloggs and get access to anything except the stuff Fred chooses to store on Google).
3) It complies with EU data protection legislation so our children's data is guaranteed to never leave the EU and its data protection bubble (P.S. Last time I asked, Apple couldn't guarantee this for iCloud and Microsoft only just got it for OneDrive etc.)
4) You don't need to set up remote desktop, or provide copies of your MS office licences to the children (yes, we DO still have to pay for them). They can work from any browser in the world on any machine. You're not limited to Apple devices but can use them to write, edit, submit and colloborate on work, you're not limited to Windows to manage it, you're not limited to a particular browser. And at no point do you have to "download" or "upload" a file - you just work live, and can even block downloads/uploads so you instantly cut out all the virus hassle of documents coming in from home.
5) For our stuff, the data guarantees are enough, and the data doesn't really matter. The school database would never go on Google but the kids providing the answers to Section 3 from the book? Who cares?
6) Google Classroom. Press a button to make a class. Press a button to upload a document. Press a button to distribute to class. All the kids get fresh copies of the document, fill it out, and send it back with one click. Mark accordingly. Worth it's (free) weight in gold.
7) Most of our parents have Google accounts already. Sharing documents with them when we need is a breeze and we can stop the pupils sharing documents with users outside our domain and lock down to any level we like.
In the last year, our tiny little school made over 120,000 Google documents. And it hasn't cost us a penny past providing an Internet line.
The only thing I've paid for is Chromebook management licences for some new Chromebooks. For less than the price of an iPad we can give the kids a secure, locked-down, single-account-locked, proxied web device only capable of going to Google sites (and the sites we choose, obviously), under our ultimate control for every option (disabling SD card ports, etc.) with a perpetual management licence (£25), and a bunch of apps, and get ten times more out of it than we can an iPad, or even an Office licence. Setting up 40 Chromebooks took about 10 minutes. Setting up 40 iPads can take days in some cases ("Hey, we see you've tried to sign in on more than 5 new iTunes accounts into Apple devices at your IP! We're going to block you for a week and make you phone through 8 support lines to temporarily raise that limit if we feel like it after you grovel to us and tell us that you have 40 new children starting this year" - actual conversation I have had with Apple...).
I quite like Google and have an account, but I do keep my email on my own domain, never signed up to Google+, etc. but I have to say, they win at education hands-down. Microsoft comes a distant second. And Apple only learned this year that schools use their stuff, so if you buy everything only through them and meet their purchase criteria and buy everything brand new and never want to do anything complicated, iOS 9.3 actually has some education-useful features like FINALLY stopping the little goits changing the devices names against your will or signing out iTunes accounts to plug in their home ones to then install all the restricted apps, games etc. they feel like and put it back on their school account, even if you have £10k of Mac Mini servers, DEP settings, and Cisco Meraki device management especially just for the iPads. I'm not even joking.
(Score: 2) by frojack on Tuesday August 09 2016, @06:29PM
Pretty sure you could have said that more succinctly.
If it is free, and Google can't make any use of the data, then it will disappear when Google declares a spring cleaning.
No, you are mistaken. I've always had this sig.
(Score: 2) by VLM on Tuesday August 09 2016, @08:15PM
I donno about the school deal my kids use.
I did volunteer at a non-profit and they paid google for the $10/month product. Its a metaphysical argument if my volunteering cost them $10 or if they were going to have a treasurer anyway so even if I didn't volunteer they'd be out $10. The feature list is phenomenal. The bill was paid by higher organizational level and I suspect being non-profit we were not paying the full $10/person anyway. I mean we got the $10/mon product but were probably only charged a nominal fee like $1 or they wrote off the full $120/person-year as a tax deductible donation to us, I really donno.
But yeah we paid money and very theoretically if we needed support they had actual humans supporting us (which is rare with GOOG)
If you want, you can pay google $10/month for apps access and they add on some impressive features.
Your point is correct that in theory they could say "F it" and quit at any time, plus or minus contracted service. So if no contracts are longer than 1 year then they can sunset in a bit over a year if they stop selling and renewing today.
I don't know why they would stop, it seems a license to print money, but those idiots did shut down Google Reader so they can't be too bright.
(Score: 2) by frojack on Thursday August 11 2016, @04:17PM
But Google reader never had a monitization scheme behind Google reader. And they couldn't see their way to making one.
Our company used to advertise via google, and the click-fraud we endured (that virtually never resulted in sales) meant we paid google far more than the load we ever imposed. So we were annoyed at the spring cleanings, but we understand their need to cover costs. We just learned not to get too dependent on any free google service.
On the other end of the spectrum, If you can embrace the spring cleaning idea: I recently read where they are selling cloud computer instances (Preemptible VM Instances) for as little as 1 cent per hour [techcrunch.com].
No, you are mistaken. I've always had this sig.
(Score: 1, Informative) by Anonymous Coward on Tuesday August 09 2016, @05:02PM
The average household is somethinng like 2.6 persons (citation currently uncontactable, due to ddos), so more like 10m attempts. The supposedly tested for 1m per hour., which is inconcevably poor planning, given majority would peg for the 'after dinner' time slot. Well that assumption and most of their published data pages ending in .nsf being a big givaway.
(Score: 2) by bob_super on Tuesday August 09 2016, @05:04PM
If you have 10 RU in rack, you can buy Ixia or Spirent boxes which will emulate a whole lot more than 23 million simultaneous connections (plus a bunch of attacks thrown in "for teh lolz".
If you don't want to buy, I believe both offer stress test As A Service.
Going online with infrastructure which collapses under expected load isn't just dumb, it's incompetence.
(Score: 3, Funny) by cmn32480 on Wednesday August 10 2016, @01:26AM
So... um... kinda like Healthcare.gov?
"It's a dog eat dog world, and I'm wearing Milkbone underwear" - Norm Peterson
(Score: 2) by bob_super on Wednesday August 10 2016, @04:31PM
Yup, that incompetence from the people not providing the proper budget to implement a law they can't manage to overturn with a veto-proof majority.
(Score: 2) by frojack on Tuesday August 09 2016, @06:23PM
I mean, however you cut that, you're causing problems for yourselves there, surely?
Why not Census Month?
Why does it have to be done in a day? Historically this task has never been done in a day, and it didn't matter.
So someone dies mid month, and never gets recorded, A Baby is born every minute. A day late an they are missed on the census. It doesn't matter in the grand scheme of things.
No, you are mistaken. I've always had this sig.
(Score: 0) by Anonymous Coward on Wednesday August 10 2016, @03:28AM
It's school holidays, lots of kids are having sleepovers. Who is going to count them? Nobody? Everybody?
Lets plan a new school, how many kids in this area? X, half X, 2X, zero, fucked if anybody knows now.
(Score: 0) by Anonymous Coward on Tuesday August 09 2016, @07:08PM
But even 23 million online simultaneous users is not beyond the realms of ordinary in the IT world, really. There must be 23 million people online on iPlayer over the course of a day, and video streaming is a damn sight heavier and more difficult to do en-masse.
I'll note that that is a different, and easier, problem. They have a continuous demand and are resourced to that effect.
By analogy, consider the following two problems. Which would be easier (note, I'm making up numbers).
1) You are in charge of snow removal in Chicago. Each year they get about 10 days of snow over 20cm. You have a budget of $2,000,000.
2) You are in charge of snow removal in San Francisco. Typically they get no snow, but due to a fluke they predict 1 day of snow over 20cm this year. You have a budget of $200,000.
Yes, you can bypass the problem with clever reasoning of weather melting the snow or other real-world tricks, but the general point stands. iPlayer/etc has both the economies of scale and the experience to deal with a large amount of demand on a continuous basis. A one-day census has neither, nor is it "worth it" to buy a lot of infrastructure for a one-day effort.
(Score: 0) by Anonymous Coward on Wednesday August 10 2016, @02:26AM
Um The census did NOT crash. It was deliberately shut down due to the hack attempt (DOS) attacks as a precaution.