Some employers have started to use big data analysis to assist with recruitment.
Nearly half of new recruits turn out to be duds within 18 months, according to one study, while two-thirds of hiring managers admit they've often chosen the wrong people.
And the main reason for failure is not because applicants didn't have the requisite skills, but because their personalities clashed with the company's culture.
So these days employers are resorting to big data analytics and other new methods to help make the fraught process of hiring and firing more scientific and effective.
For job hunters, this means success is now as much to do with your online data trail as your finely crafted CV.
This discussion has been archived.
No new comments can be posted.
Employers Turning to Data Analysis to Assist Recruitment
|
Log In/Create an Account
| Top
| 52 comments
| Search Discussion
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(Score: 3, Interesting) by Anonymous Coward on Friday October 03 2014, @11:43PM
It's really easy to hire programmers these days. I don't think "big data" or any junk like that is needed, at least when it comes to this kind of hiring.
I just ask each candidate two questions:
The answers to those two questions tell me everything I need to know about the candidate.
The exact answers themselves aren't overly important. Rather, it's the general theme that matters.
A good candidate will dislike both of them. He or she will usually point out how broken JavaScript is, and how obnoxious the members of the Ruby community tend to be. If the candidate also voluntarily says something about NoSQL being bad, this is the person you want to hire immediately. They have knowledge, they have skills, and they have experience.
A bad candidate will like both of them. Chances are this fellow (it's almost always a man; women tend not to hold this opinion for some reason) will call JavaScript a functional programming language, or say JavaScript is just like Scheme, or he'll downplay Ruby's horrendous performance, or say that Rubyists are talented, or make some equally stupid and baseless assertion. I know that he probably isn't qualified for the job due to a lack of knowledge and experience.
A mediocre candidate will either be neutral about them both, or like one but not the other. Unless you're desperate, a better candidate would be preferred. These people probably aren't total disasters, but they aren't the best there is.
With those two simple questions you sure can tell a lot about a programmer.
(Score: 3, Insightful) by jcross on Saturday October 04 2014, @12:06AM
That's an interesting strategy that I will have to try out. It would seem to filter out people who jump on whatever is hip at the moment. I've trusted decisions made by such people and ultimately regretted it in a big way. I mean, something can be trendy now and wind up continually improving or having staying power for some other reason, but the problem is you can't really tell in advance whether that will be the case. So personally I would be looking most for a nuanced response, especially with Javascript because, however much it sucks, if you want to deploy on browsers you're pretty much stuck with it (or something that "compiles" to it at least). I think the ability to make an imperfect technology shine as best it can is a valuable and under-appreciated skill, and beats wasting energy fighting or dissing it.
(Score: 3, Insightful) by fadrian on Saturday October 04 2014, @01:01AM
I think you're leaving out one important class of people - people who understand both the pros and cons of each of these (as well as other technologies) based on problem and business context and can discuss the tradeoffs their use brings clearly and rationally with you. Because technologies are not Manichean. There are pluses and minuses to each one's use depending on context. I'd think you might want someone understanding that, given that technology... you know... changes?
That is all.
(Score: 2, Funny) by Anonymous Coward on Saturday October 04 2014, @01:15AM
The beauty of those particular questions is that there are no pros at all, and there are only cons. But some people, namely those who aren't qualified, only see perceived pros, without noticing the cons.
There's nothing good about JavaScript. Everything about it is bad. Nobody can responsibly hold a good opinion about it.
It's just the same for Ruby. There's nothing good that can be said about it or its community.
That's why those questions work well. Those who fail the test have legitimately failed it. Those who pass it have legitimately passed it. Anyone in the middle should probably be considered to have failed it.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @03:18AM
There is nothing good about a hammer. You cannot even make a computer game with it.
There is nothing good about Python. It does not even tell you how far apart you should plant potatoes.
Neither can it turn water into wine, nor make my herpes go away.
Oh look, your language of choice is also imperfect. https://wiki.theory.org/YourLanguageSucks [theory.org]
Shall we compare the magnitude of suck (which is multi-dimensional) instead of making proclamations?
The one that really stands out is PHP: http://blog.codinghorror.com/the-php-singularity/ [codinghorror.com]
Also, you are kind of stuck with JS if you want to do anything remotely complex on the web. Dart language is surely cleaner, but is there a magical js2dart command that works 100% of the time, including MY legacy JS-powered web UI project? Will it take off and justify my investment, or will I get stuck with a yet another dead JS-killer-wannabe?
There is something to be said for wishful thinking, but outright masochism is unhealthy. Some of us need to do some web UI work during the day without having to resort to drinking in the evening (due to family commitments) , so we choose to forget and ignore.
(Score: 2) by fadrian on Monday October 06 2014, @03:51PM
Dude, it's Turing complete - in fact, it's pretty much beyond that. Yes, it has some pretty fucked-up pieces (show me one language that doesn't). But it's also one of the most widely-used languages (making programmers who know it easy to find), close to ubiquitous in its deployment (so it's available on most systems via almost all browsers - I'd guess lynx and other text-based browsers still doesn't have JS support), has first-class functions (even if scoping is a bit messed up), and a wide variety of libraries that are quite suitable for its purpose. Those are positives, even if you don't want to acknowledge them.
One thing I've learned in this world, programming on everything from IBM 360's to AS-400's to engineering workstations of the 1980's to PC's to VM's in a variety of shells, editors, languages, and companies is that there's no computer technology that can't be used. I think you're looking at your own prejudices and trying to make sure everyone you hire fits into your own, oddly narrow point of view. Sad. And ultimately self-defeating.
That is all.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @02:44AM
You started sort of in the right direction, but I want to point out that declarative statements do not an argument make. I do not like JS myself and have no reason to switch to Ruby from Python, hence I only heard about Ruby being slow, never experienced it myself and I can neither confirm nor deny that Ruby programmers smell. I have not interacted with them, and even if I did, it would take me awhile to form an opinion of this large community backed by solid statistics.
You would need to consider where people are coming from. It is a safe bet that they are in many ways different from you, have experienced different experiences.
If they have zero Ruby experience (and they could be amazing coders), then they would likely not have an opinion.
If they coded in Ruby for over 5 years, then they probably like it regardless of the arguments you present to them. Instead of inviting them to an interview just so that you tell them that they are part of a group that is a munch of borons, you could have culled them out sooner and saved both of you a lot of time.
Only if their DNA is like yours and their education & career was very similar, is there a good chance that they will think like you.
In my opinion, a better approach is one outlined by a young (but nonetheless accomplished) blogger/startap-er Ted Dziuba. He asks them what pisses them off about the tools they have used in the past, and tries to judge the depth of their knowledge/amt. of gained experience from that.
http://widgetsandshit.com/teddziuba/2009/12/how-i-spot-valuable-engineers.html [widgetsandshit.com]
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @03:09AM
Ruby I can see as pointless, but how exactly does a job candidate get away with condemning Javascript when there's basically nothing to replace it for universal client-side scripting? Javascript has its flaws, but it's also ubiquitous in good web development (i.e. everything beyond gopher). It's a necessary tool. That doesn't mean developers need to "evangelize" the latest hip library, but they shouldn't just reject the "mainstream" either out of hand. Like someone else said, I'd want a more nuanced response, but then I'm just a chipmunk.
(Score: 1) by Gertlex on Saturday October 04 2014, @03:43AM
Easy: he gets away with condemning it by then acknowledging that we're stuck with it for far longer than anyone honestly would like.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @04:03AM
There are many candidates who would prefer to not ever touch javascript. There are also many candidates who would NOT want to work for an employer who rejects javascript like that.
So perhaps it is a good idea to ask your showstopper, no compromise possible questions if you can ask them in a way that you can get an honest answer (lots of people lie) and you don't get in big trouble (sued, fined, prison etc).
For example, if you don't want the extreme jewish/muslims in your organization, you might ask them if it would be a good idea to ban bacon cheese burgers etc from menus for work-related functions/events. Same for christian nutjobs - go bring up creationism vs evolution. Or the feminazis - ask what they think of hostile environments like colleagues having scantily clad attractive ladies as their desktop wallpapers. Or the KKK - ask them about how'd they feel about blacks (depending on whether you're one of the KKK, shoot them is a plus/minus points answer ;) ).
Yeah you're discriminating against people for their beliefs. But guess what, beliefs affect that company culture thing which the story is talking about. Why waste months of everyone's time (theirs and yours), if they don't fit in AND nobody is willing to compromise.
The tricky bit is how to find these things out without getting in big trouble ;).
FWIW I know someone who was a boss in a muslim majority country and said he avoided a lot of problems by only hiring "muslims" who were fine with drinking alcoholic beverages. He'd probably get in trouble in the USA for asking such questions...
(Score: 1) by khallow on Saturday October 04 2014, @10:48AM
You had me going until you misspelled "LISP".
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @12:15PM
Haven't you heard? JavaScript is a dialect of Scheme. Ruby is a dialect of Common Lisp. They're functional, man!
(Score: 2) by maxwell demon on Saturday October 04 2014, @12:10PM
Good, as in sharing your bias?
While it has definitely its quirks, JavaScript isn't exactly a bad language. It's just that the vast majority of people learned it from imitating bad code, which means that ultimately 99% of all JavaScript code is bad.
For example, it is still a widespread misconception that JavaScript doesn't allow to define private object members.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @12:19PM
No, JavaScript isn't a quirky language. It's a fucking busted language. There's no such thing as "good JavaScript code" that people can learn from. The language is so goddamn busted that such code can't even be written. It's not just 99% bad. It's 100% bad.
While JavaScript may allow private object members, it's usually done in a totally hackish way. It's not like Java, or C#, or C++, or other sensible programming languages where you just use the "private" keyword. In JavaScript you need to hide it within a local variable in a closure or some similarly half-arsed way of doing it. It's not an intentionally designed feature of the language, but instead it's just an accident that it's even possible.
(Score: 2) by maxwell demon on Saturday October 04 2014, @12:28PM
So doing anything different than your favourite language means being "bad" and "hackish"? One could just as well consider the public/private concept of C++/Java/C# a hack to work around the lack of closures.
And I'd like to have that mind-reading technology you use to read the JavaScript language developers' mind ;-)
The Tao of math: The numbers you can count are not the real numbers.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @07:25PM
It's not about JavaScript being different. Being different is, well, totally different from being irreparably busted like JavaScript is. SML and Haskell are different from C++, Java and C#, but they aren't busted and moronic languages like JavaScript is.
And you clearly don't know what you're taking about, as I have discovered. C++, C# and Java do have closures. But since they are properly designed languages, visibility is a well-defined part of those languages, rather than an accidental offshoot of having closures.
(Score: 2) by maxwell demon on Saturday October 04 2014, @07:45PM
C++ (and especially private members) existed long before C++11. Also early versions of Java didn't have closures. I don't know about C#, admittedly.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 5, Insightful) by Ethanol-fueled on Friday October 03 2014, @11:44PM
Their "personalities clashed" with the company's culture?
As in, they bit back in the face of relentless bullying and abuse, didn't want to work more than 50 hours a week, didn't want to have to fight every other department to get the most menial tasks done because metrics are set up to discourage cooperation? Because they dared to suggest a better way to their boss or worse, their boss' boss? Because they refused to let others take credit for their own good ideas? Because they told self-proclaimed delagators and other budding tyrants to fuck off and do their own damn work? Because they'd rather eat lunch by themselves then reveal their darkest secrets to the nosiest and noisiest backstabbing gossips? Because they'd rather keep their politics to themselves? Because they were religious in San Francisco or gay in Alabama?
It's a wonder that more corporate organizations don't get a "personality clash" in a back alley by packs of disgruntled ex-employees.
(Score: 3, Interesting) by Anonymous Coward on Friday October 03 2014, @11:53PM
I'm glad that you mentioned the "religious in San Francisco" part of this problem. For an area where the people claim to be "tolerant", "accepting", "progressive" and all of those things, they sure can be the most bigoted and intolerant people around if you don't exactly fit into their mold of what's considered "socially just".
You don't even have to be that religious to get on their bad side. You don't even have to be religious at all. If you don't exhibit the so-called "social justice warrior" mindset 150% of the time or more, they'll label you a "racist" or a "sexist" or "transphobic" and treat you like trash. Even remaining neutral with regards to something like homosexual marriage will result in them condemning you for being an "intolerance enabler".
It gets really disgusting when they try to infuse their ever-changing social "cause" of the hour into absolutely everything, even when it's totally unnecessary. No, I don't want to buy a goddamn plastic bracelet from you to "support" whatever useless cause you're "promoting" for the next ten minutes, until the next stupid "cause" hits YouTube!
(Score: 1, Funny) by Anonymous Coward on Saturday October 04 2014, @12:06AM
It should give you pause when you find yourself cheering on e-fueled.
(Score: 1, Interesting) by Anonymous Coward on Saturday October 04 2014, @02:59AM
I tried explaining why I am a pro-gun libertarian / anarchist and a prepper to a group of liberal coworkers in Massachusetts who in the midst of 2012 election campaign would watch Obama's speeches and then recite/analyze them the next day during lunch at work and comment on how smart Obama is and how inspiring his speeches are.
Let's just say I failed to convert any coworkers to my religion of choice.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @03:17PM
Extremism is bad no matter what belief is held.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @06:05PM
The truth is extreme. To make it moderate is to lie.
There was a time when asking doctors to wash hands was considered an act of extremism.
Judging validity of something based on how much it deviates from current believes rather than evidence behind it is a form of logical fallacy.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @09:15PM
It was not considered extremist, it was considered crazy; the very idea that little things you couldn't see could make you sick was insane! The people who are not open to new ideas, firm-held in their beliefs that refuse to accept any new knowledge, are the extremists, regardless of if they're the majority or minority.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @08:02PM
Being responsible and self-reliant isn't "extremism". It's the most normal, non-extreme thing any living organism can do.
Those who seek to build large social structures that crush individualism are the ones engaging in extremism.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @09:10PM
You think I was calling the pro-gun side in that anecdote an extremist? While its not an extremist view, it clearly comes with a persecution complex.
(Score: 1, Insightful) by Anonymous Coward on Saturday October 04 2014, @06:54AM
To clarify, what they meant by "personality clash" was that the people they hired actually HAD personalities instead of being soulless automatons silently awaiting the next mundane yet contraindicative order to be hurled down upon them by the collection of drunken, nepotistically gifted business school graduates otherwise known as "senior management".
(Score: 1, Insightful) by Anonymous Coward on Friday October 03 2014, @11:51PM
The problem with applying Big Data analysis is the "self-fullfilling prophecy" - we start making decisions based on the statistical analysis and it creates a feedback loop that just reinforces the status quo and keeps progress in check. That might make your short-term metrics look great, but the long-term ones like continuing growth will end up stagnant. Companies that embrace this sort of statistical determinism will end up frozen in time while the rest of the world leaves them behind.
(Score: 3, Interesting) by Anonymous Coward on Saturday October 04 2014, @12:09AM
This is a real problem for sure, I do believe.
I wonder how much this sort of data collection and analysis has resulted in the downfall of Firefox, for example.
Firefox can collect usage data and send them back to Mozilla, where they're presumably analyzed. In the distant past, back in the Netscape days, this was mainly just bug reports. But more and more types of usage data have been collected over time.
As more and more data have been collected, the Firefox user experience has gotten worse and worse. I can't say for sure, but I would not be surprised if the data Mozilla collects and bases their decisions upon only encourages more bad decisions to be made.
I am curious to know if it works like this: The data causes Mozilla to make some bad decisions. These decisions ruin the Firefox experience. The users who dislike these changes move to Chrome. The remaining users end up generating misleading data. More bad decisions affecting Firefox are made. A new release comes out with these changes, and more angry people move to Chrome. The new data are collected and incorrectly suggest that the changes were well received, when they actually weren't. And it cycles on and on, getting continually worse.
It's easy to say that something like Australis was "well received", if say 75% of 1,000,000 users didn't totally hate it. But it's misleading to trust those statistics when, say, Australis itself drove away 30,000,000 users, all of whom hated it completely.
(Score: 1, Insightful) by Anonymous Coward on Saturday October 04 2014, @01:20AM
economic statistics are similarly misleading, such as government inflation figures, which are based on a (very) select bag of goods that are deliberately selected to make inflation appear to be in check. when one of those products inflates too much, they remove it from the equation.
also, similar distortions in economic statistics occur when government intervenes in an otherwise free-market economy...
for example, in australia the government 'stimulated' the economy during the gfc by offering free home insulation if certain conditions were met
this lead to an artificial surge in demand for home insulation installation, which in turn lead to a surge in demand for insulation batts, etc (good right?)
companies hire a bunch more people to meet said demand, manufacturers ramp up their production chains, acquire more machinery (usually with bank loans)
problem is, when said stimulus is removed, then what happens?
... a bunch of people lose their jobs and companies go broke cos they cant repay loans for equipment they never really needed in the first place
that's why its called 'stimulus'... because its like a short-term high, followed by long painful withdrawal symptoms
despite all this, you can be sure the statistical numbers looked awesome when the stimulus was going on, which is all that many people seemed to care about.
/rant
(Score: 2) by sjames on Saturday October 04 2014, @08:48AM
They knew it was stimulus and that eventually the program would end or the qualifying houses would run out. Why did they 'plan' on the bubble lasting forever when they had every reason to believe otherwise?
(Score: 3, Interesting) by mhajicek on Saturday October 04 2014, @05:04AM
This reminds me of the WWII question of how to up-armor aircraft. The first guy said to look at the planes that came back with holes, and put armor where the holes are. The second guy said to look at the planes that came back with holes and put armor where the holes weren't, since planes that got holes there didn't come back.
The spacelike surfaces of time foliations can have a cusp at the surface of discontinuity. - P. Hajicek
(Score: 1, Insightful) by Anonymous Coward on Saturday October 04 2014, @07:05AM
Agreed;
The problem with applying Big Data analysis is the "self-fullfilling prophecy" - we start making decisions based on the statistical analysis and it creates a feedback loop that just reinforces the status quo and keeps progress in check.
And this goes exponential once it gets applied to hiring hiring managers, or hiring CEOs .. Brave New World, much?
(Score: 4, Interesting) by jcross on Friday October 03 2014, @11:57PM
I doubt big data is going to help with this problem; it's just not automatable. What you need is someone who's a good judge of people, and then a way for them to efficiently evaluate a lot of people, but it's hard to do this with a high throughput. For instance, I can decide whether a person is going to work out socially in five minutes or less, and since some important learning experiences I haven't been wrong. With good management (which is to say effective and minimalist management), a decent product that developers can find their way to caring about, and by selecting the right people, you don't need the process to have a high throughput, because a small team of this kind can accomplish amazing things. Unfortunately most hiring managers won't be able to create these conditions, since the company culture is out of their control, and can only overhire from the huge pool of mostly mediocre skill and overpay because they're offering such a crappy work experience. In this situation it's nice to think big data would help, but I think by the time you're considering that it's probably too late to fix the foundational problems.
(Score: 5, Insightful) by TheGratefulNet on Saturday October 04 2014, @01:51AM
almost every boss I've had and almost every dev I've worked with in the past 30 or so years (I'm over 50) has been INCAPABLE of truly doing a good interview.
they ask coding questions. they ask you to write code on the board. they ask you to recall things as if you were a disk drive. they do things ALL wrong.
lots of people like me can do great engineering work but I mostly get passed over when I go onsite. they have no idea how to deal with the outliers who do NOT memorize things, who do just-in-time lookups for algorithms and adapt them, who do JIT things for pretty much everything these days. I have too much stuff to know about (hardware, software along makes memorization nearly impossible; chips, circuits, software, os's, frameworks, test equipment - the domain is so wide, you HAVE to flush shit out of your memory just to be able to do a varied job these days).
and yet, the clueless managers and 'programmers' keep on doing the same interview style and they wonder why they get the wrong people.
its really frustrating. but I see nothing that is going to change it. software is becoming a sweat shop and a race-to-the-bottom for youth and low pay (and h1b). until all of this changes, you won't get any meaningful change in the interview style or technique. they will continue to roll the dice and they'd actually be better off just doing a random hire than putting people thru the seive like they do now.
google admitted their 'genius' style interview was not getting them the right kind of people. it took them years to admit it. still, I doubt they have changed in any meaningful way. that's just one example.
one time out f 50, say, you'll get a manager who does not play those games and you'll have some older engineers who also don't play those stupid 'tell me what parameter #3 is in this API call' games. but its rare as hen's teeth and when I find a job that has people like that, I realize how good and rare it is and I try to stay there as long as I can.
most tech companies are useless in how they hire. stop letting teenagers (so to speak) run things and you'll get the wisdom of age on your side. you'll see a totally different level of people coming in if you let the adults do the decision making. but again, that's not how current tech companies are run.
"It is now safe to switch off your computer."
(Score: 2) by tonyPick on Saturday October 04 2014, @09:37AM
almost every boss I've had and almost every dev I've worked with in the past 30 or so years (I'm over 50) has been INCAPABLE of truly doing a good interview.
Speaking as someone who has done interviews in the past I'd agree, and admit that *I'm* crap at it (but still get asked to pitch in).
The problem (as I see it) is that having someone who is technical enough, and has the right people skills, to correctly evaluate a candidate in an hour or two interview slot is very (very) rare. If you find someone with the right skill set to do that then they've got the right mix of skills to (say) put them in front of customers, talk to suppliers, handle team interaction, and run as technical lead on projects.
So if you get somebody like that (and they're rarer than rocking horse sh1t) you don't want to have them doing the interviewing, and *they're* fairly unlikely to want to do interviewing full time themselves.
Plus getting a full time position being paid as much as a senior architect, and all they do is interview people and reject upwards of 90% of the candidates? Management would have a heart attack over that ("Can't we just come up with a standard set of questions and have Joe the typist run through it with them?")...
So I don't think this is even the race to the bottom, even though that doesn't help, it comes down to "not enough good people in the world".
(Score: 3, Insightful) by VLM on Saturday October 04 2014, @01:02PM
stop letting teenagers (so to speak) run things
MBAs, more specifically CYA culture.
If you rely on judgment and get it wrong, boss has to be fired. If you just stack rank who got the most answers right on a API memorization test, nobody is to blame so nobody in mgmt needs to get fired.
So everyone who hires sensibly gets selected against until you only have the masters of CYA left.
(Score: 2) by kaszz on Saturday October 04 2014, @11:48PM
Wisdom doesn't come like magic from age. It's what you are, think and actually do with your life. Some people are retired and are still not wise..
(Score: 2, Insightful) by Anonymous Coward on Saturday October 04 2014, @02:50AM
For instance, I can decide whether a person is going to work out socially in five minutes or less
My charm shield can do wonders when it is on. It only lasts about 2-3 hours per day, just long enough to pass an interview, and then I become myself again.
(Score: 0) by Anonymous Coward on Friday October 03 2014, @11:59PM
The biggest danger of Big Data is that it brings NoSQLers into an organization.
Professional DBAs and data analysts, who have experience working with database systems like Oracle, DB2, Informix, PostgreSQL, SQL Server and Sybase, don't really have the concept of "Big Data". To them, terabytes, petabytes and even exabytes of data are routine. They're all just "data". There's no need to qualify even just a few hundred gigabytes of data as "Big Data" like we see the NoSQLers do.
This happens because NoSQLers tend not to have even the most basic of data warehousing or data analysis skills and experience. Even small problems seem huge to them.
Successful organizations don't have the concept of Big Data, because they've only ever hired qualified professionals. "Big Data" really means "Big Problems", and it's because the people who push "Big Data" usually don't have a damn clue what they're doing.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @01:30AM
What is sad is every word you said is true. But it is also false. The truth is more in the middle.
My company desperately needs something *like* nosql. But then they put in requirements that make it impossible to do (basically realtime reporting). Then they do not understand why it does not work. Because you are inflexible on your requirements that you made up. You have not even bothered to ask our customers if they would relax on time until complete. They then end up trying to do 'nosql' like things in a ACID database and can not understand why it does not perform correctly. Or suddenly the query does not work out correctly (yeah you removed the locks and transactions).
NoSQL has its place. It is brilliant in things where 'if the data is a little off for a few mins' and you want crazy high insert rates. It blows on data retrieval and query. It makes sense for things where you want brilliant insert rates then a secondary processor to unwind the transactional data. You can do these sorts of things in SQL databases (I do it every day). But it takes a large amount of time and care to get 'just right'. I am not talking 10-20 inserts a second. I am talking 20-50k a second.
It works in places in aggregation and pre SQL steps. Unfortunatly no one really wants to use it for that.
This happens because NoSQLers tend not to have even the most basic of data warehousing or data analysis skills and experience. Even small problems seem huge to them.
Its not just the nosql'rs that make that mistake. I have seen enough db projects do the same thing. Report data right on the transactional data and no roll up data then wondering why the reports take forever and a day to run. Then management is 'too scared' to let anyone change the database so you cant even roll it up even if you wanted to.
(Score: 1, Interesting) by Anonymous Coward on Saturday October 04 2014, @01:43AM
for feedback.
Too funny.
I got laid off from a company that was into big data and even tried to apply it to hiring process.
I was quizzed on the algorithms and trick questions to get accepted. I did well on that, partly because I have been to dozens of interviews and half of them do this exact thing. I also could not ask them too many things because whatever they were doing involved a lot of secret sauce and non-disclosure. Also, my headhunter was too good and what he did and managed to manipulate me (an INTJ) into taking something that was not ideal for me.
When I later asked them if they ask candidates questions actually relevant to day-to-day job, I was told that they look for candidates that can think abstractly.
Few months down the line I was forced to switch from dev to devops/support position. For the fun of it I actually wanted to give it a try.
I sucked at it for the same reason I could not get a B in my non-technical college classes - my memory is crap. I have to rely on understanding instead.
Not what devops/support (at least their version of it) required. I needed to memorize a hack on top of hack and how to hack one other hack.
The guy who wrote the code still remembers what he was thinking and what he had for lunch 3 years ago when he wrote these lines. I bet he even knew how to fix this cleanly, but he got promoted and became too busy to do development anymore. As for me, I tried taking notes and writing things down, but the hacks to work around were too numerous, and I was not allowed to take my time to re-write and test the code.
Instead of throwing puzzles at me, they should have asked me to recite the non-disclosure agreement back to them perfectly after a single pass of reading. I would have failed; they would not have hired me.
Then I guess I pissed a colleague off when he complained that some people take the last can of soda and do not put the full case back into a fridge. Gawd, a 1st world problem!
I thought to myself: When there are so many choices of relatively healthful drinks available to us for free, why would you go with soda? Regular soda has too much sugar, and sugar is not good for you (particularly you). Diet soda is even worse, the artificial sweetener breaks up inside of our body at mere 90F into some rather nasty chemicals.
I could have said that verbatim, but I am an INTJ, so I said: "You could just not drink soda." He returned an evil grin.
I also saw through the CEO's BS and paraphrased it when he gave a smooth speech, essentially rewarding employees with a lot of "thank you" and "you should be proud of your accomplishments" and "you are changing the world" instead of dealing out cold hard cash.
When hiring, they could have asked me for my Myers Briggs personality type. I would have told them that I was an INTJ. They would not have hired me. End of story.
When I was told something along the lines "today is your last day, you do not fit in", to their surprise my reply was a cheerful "I agree".
I also could have given them a ton of constructive feedback on how/who not to hire, but I was given about 10 minutes to collect my stuff and leave.
Point of the story - forget the big data, just ask for some feedback.
On the second thought, that can backfire. Our litigious culture can be hard to navigate.
P.S. Thank god they got rid of me. I now work less and earn more.
(Score: 2) by goodie on Saturday October 04 2014, @01:56AM
Ok, I'm gonna take a stab at it and say that the time I posted this message, you have posted and replied to most of this thread ;).
Got an axe to grind :D?
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @02:14AM
Sorry, not trying to be a dick (or trying not to be a dick), but I did not grok your reply (as written) 100%.
Perhaps you omitted a key word.
Let me try to guess what you meant ...
Firstly, I am new here.
Should I have replied to an existing thread instead?
Maybe, if I saw one that was very relevant to what I wanted to say.
Either I missed it or it does not yet exist. My thread is one of ten or so.
Are you trying to say that I am less special than the other 10 trend setters?
Do I have an axe to grind? Initially I did. I am certainly over it now, but I do feel even better after posting.
I think my story has value to others. Yet another reminder that interview should be a 2-way street but most often is is much closer to 1-way in practice (They grill you. They decide. The end.)
Also, I am convinced that about half of the people who conduct interviews have no idea how to do it properly.
One gave me too many hints before I had a chance to try a problem myself. I could have been half as smart and I still would have passed.
Effectively, it looked as if he wanted to show me how cool (smart+nice) he was rather than try to weed me out.
I was there too. For the first 3 years of my career I was terrible at interviewing.
No wonder that such a high percentage of hires do not work out.
P.S. Also, regarding the diet soda. I did not make it up and I did not read the article right below before I made my post.
Bad news about diet soda circulated for some months now.
http://soylentnews.org/article.pl?sid=14/10/03/1221228 [soylentnews.org]
(Score: 1) by new here on Sunday October 05 2014, @12:34AM
No, I am new here.
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @07:40PM
You bring up an excellent point. Many companies today (all that I have worked for) have hiring processes that reward Extroverts. Quick answers are rewarded. Out loud brain storming on a completely new problem, on a white board. Etc. These are skills that Extroverts excel at and Introverts do not. All the companies claim they are not biased towards Extroverts but it's just not true. Feedback from former employees would help, but that feedback is never used, even when it is asked for. Performance reviews at my last company were also highly favorable to Extroverts. To play the game an Introvert must literally "play" a game, day in and day out. And that is one big reason there is high turn-over in tech organizations. Start-ups are even more biased towards Extroverts - again, at least the several that I have been involved in/with. Using "Big Data" could help the companies root out the Introverts early and find more candidates that fit their culture, but that won't be a good solution for them in the mid-long term.
Sadly, many great programmers will bounce from company to company until they find one that values what they offer - but those companies are rare. Keep looking is the best advice I can give.
(Score: 0) by Anonymous Coward on Sunday October 05 2014, @09:30PM
While you appear to agree with me overall, I will disagree with you on details. Basically, I can get along with certain extroverts just fine (hint: the very "smart" (accd to me) ones).
I am going to assume that Myers Briggs personality typing is a lot closer to science than to astrology. I do not know nearly enough about why it works, but it certainly appears to work for me.
I will further assume that the majority of programmers come from a "rational quadrant" - they are NTs. Feelers (as opposed to Thinkers) would not generally do so well in this capacity ans Sensors (as opposed to iNtuitives) would more likely be the IT equivalent of a mechanic rather than mechanical engineer. There is room for that in IT, perhaps in QA (generalizing)?
The extroverts that are NTs are not that bad. Well, an ENTJ can come across as an asshole and someone who is not as deep a thinker, particularly when they are young. ENTPs tend to be the fun loving type who will chat during lunch for sure, but they also know how to shut up, sit in front of the screen and get some work done.
Startups probably like ENTJs because they are all about doing stuff and not over-thinking. This is probably a good trait in a start-up environment. Startups can be gruelling and not for everyone.
Now, if the company does not produce software as their primary product, or even if it is a specialized software targeting some extroverted industry, then there will probably be all sorts of non-technical characters and it could easily be ruled by the extroverts.
In a real IT shop the relative frequencies should be about the same. http://www.capt.org/mbti-assessment/estimated-frequencies.htm [capt.org]
I was mentioning my type in particular - INTJ. We are the least flexible, the most stubborn SOBs of the 16 types. Being a conformist or keeping one's mouth shut when there is an opportunity for (what appears to me as) constructive criticism is literally hard and painful, like having blue balls. Not saying something when the duty calls for it is like not letting that protein out when it is ready to go. INTPs and ENTPs are more mellow in that regard and ENTJs gravitate toward management. So, in part the trying to fit in is more specific to an INTJs than any other introvert. With that, I do not know how useful this post is to others, I will admit that this is partly a rant.
I agree, performance reviews is a dehumanizing experience. I am finally a contractor and do not have to deal with those.
Keep looking is the best advice I can give.
Amen to that!
Now, as far as interviews in general and the white-board problems - I tend to do ok on those because:
A) I am in general confident, if I make a mistake I can rationally think "oh well, I lost one point", let's keep going and see what happens. The upside is large.
B) I do well on timed tests in general, unless time is so short that the only way to succeed through an analytical problem is to have memorized the answers. I do not think that makes me a bad engineer, I just happened to do well on timed tests and I can create useful software (IMO).
C) I have worked at half a dozen companies and have been to dozens of interviews. If I fail technical interview, I then try to nail those problems at home. A lot of companies tend to care about the same stuff - some data structures, some algorithms, recursion, pointers, deadlocking, multi-threading, design some system and yes - puzzles. I can frequently do well on these questions because I have seen them before, and some of them matter in my day-to-day coding. I have heard weird ones like - tell me all about unit testing you do and how you do it and how you approach it. Well, in some dynamic languages you will need a ton of unit testing. In C# I could both rely on compiler and use functional programming features to make bugs less likely to appear, plus we had QA, so testing was less important.
What am I getting at? There are thousands of dysfunctional companies out there where being an introvert is not the only problem to have.
In a real, well-run IT shop I do not think being an introvert has any significant disadvantages.
Going back to dystopia - instead of using big data, if most companies would just stop sucking, they would not be having those revolving doors.
My verbose 2 cents.
(Score: 2) by kaszz on Saturday October 04 2014, @11:56PM
"I now work less and earn more"
How did you accomplish that? or what circumstances did you change?
(Score: 0) by Anonymous Coward on Sunday October 05 2014, @10:03PM
Ok, there are several factors, luck was one of them:
1) I was actually underpaid for a while. Also, I have been slowly but steadily growing over time, trolling StackOverflow for answers, watching relevant coding videos, reading a couple of books. I have not gone to coding meet-ups, partly because of introversion (I think I would gain more by watching a recording of a meet-up), partly because of time constraints, but I plan on starting that once I organize my life better.
2) When I was forced to look for work again, I did so at the tip of the market (as in I think it is about to collapse like it did in 2008, but anyhow ...).
3) According to me and my mother, I am a good engineer. Some other people also agree.
4) The last underpaid job that I took - I was a bit out of my depth there (but the employer knew that). I strategically decided to gain some new skill on a job. I learned quite a bit, but I was not super-productive initially. It just so happened that my new gig required pretty much the same skills, and I was already much better than a noob at it.
5) Getting paid as much as I am worth was not the hard part of my job search, the hard part was maintaining freedom - e.g. not hating work and not working too much. This is where luck comes in.
6) I found a new job through friends.
7) My new job is in a profitable company, it is actually a small, specialized group within a large company with reasonably good culture. I probably would not like it as much if I were in a different department.
8) My manager is just awesome! Statistically, this is bound to happen to someone. I am proud to be part of this statistics.
9) I work as a contractor from home. I can start working past 1 pm if I wish. I average 30 hrs/week :) I only get paid when I work and on the flip side I never work for free. I can afford to work less - single guy, no kids, no mortgage, barely any loans, rational spending habits and yes, a salary that I deserve, probably a bit more.
10) Well, my company is somewhat far from where I live, so by my standards I am slightly overpaid, but by theirs this is probably not the case.
11) I still make a lot less per hour than IT dudes on Wall Street or in Silicon Valley, but I think I am an order of magnitude more free than they are.
12) A catch - occasionally I have to travel. In this particular case I do not mind.
Life is too short; my advice to you is - try to figure out what would make you happy and go for it and make it happen. I think being a contractor is a good way to do it. It involves more paperwork and more discipline, but once that becomes a well rehearsed routine it gets easy.
Finally, network, network, network!
I am an INTJ. My social circle is very small, but those who remain around me are positive, smart, mutually useful. It took me awhile to build that and I am still slowly building. You see, if you follow your heart outside of work, if you pursue the exact hobbies that YOU like (not trying to please a relative), then you will eventually run into people with whom you click. Some of them will be very pleasant to be around. Some of them would like to work with you. One might even start a company from scratch, and imagine that - an entire company could (at least initially) consist of people that you like working with and be around.
I think it pays to be creative, persistent & picky when it comes to both finding a partner and a job. It is a lot of effort upfront, but the dividends will be paid back daily and hopefully for a long time.
Good luck!
P.S. If you have an amazing idea, try Kickstarter or a site like this. Imagine raising 100k to work on your favorite hobby!
(Score: 0) by Anonymous Coward on Saturday October 04 2014, @06:55AM
computer fails at nuanced social problem. I know, let's throw more computer at it!
aka quantification as CYA for hiring incompetent managers. but the computer said person X would be great here! not my fault at all.
(Score: 2) by PizzaRollPlinkett on Saturday October 04 2014, @10:23AM
Recruiting is already a dysfunctional process. I think recruiters are the Wal-Mart door greeters of the white collar world - people who could not get jobs doing anything else. They're not the sharpest knives in the drawer. And recruiters don't have access to some hidden pool of talent companies don't know about. Recruiters attract bottom-feeders for crappy jobs. No wonder most candidates don't work out - you don't need big data to tell you what's wrong. So now companies want to further dehumanize it more? Maybe the dysfunction is from dehumanizing the process in the first place.
(E-mail me if you want a pizza roll!)
(Score: 2) by kaszz on Sunday October 05 2014, @12:01AM
The HR department needs to expand and lobotomize the working environment even more! The new sign will read "HR - Human Raping" for profit & profit and CEO ego ;)
HR are there to make sure applicants have irrelevant merits so they later can fire people for being redundant when the quarterly shareholders demand more profit.