This week, reader “Earl” tells us that just this year he responded to “a Craigslist ad for a Novell NetWare Admin to figure out why .nlm files would not be loaded and fix the issue.”
[...] The return call came “almost instantly” and Earl “gave them my expensive price and advised them that I was not the first choice for a NetWare admin, but I had extensive system troubleshooting experience.”
Those caveats didn't matter: the person who placed the ad said he's run it for months and months and never had a reply from anyone in the USA. Earl was just 90 minutes away by train and got the gig.
When Earl visited the site, he was told that an electrical storm had taken out the NetWare server and Windows 95/98 clients. Said server was a Dell PowerEdge 1300 with 64MB of RAM and a 10GB IDE hard drive. Earl reckons it was built in 1997 or 1998, so was a bit taken aback when told this was “the new server”.
[...] Next came a request to boot up the Compaqs, which had power supply and fan failures. A request to swap the disks from the dead Compaqs was not something Earl could do, as they had tossed out the necessary SCSI cables a few years back.
Earl was asked to do all of these things so the company could run its bespoke accounting program, which was written for it in 1993.
The developer, it turned out, had died in 2001. But the source code was in the company safe … on about 2000 pages of dot matrix printer paper. And there were backups of the old data … on 20 years worth of floppy disks and a pair of CD-ROMs.
[...] Earl told the company that they'd need a working server, running NetWare, before he could even begin to contemplate the task of typing in the source code so he could see if the backups could be restored. Then he'd have to hope that a Pascal compiler could cross-compile for NetWare to have even a chance of setting things to rights.
To the company's credit, it tried hard to meet his requests. Two weeks later Earl says he returned to the company, where a working PowerEdge 1300 with a PCI network awaited.
[...] But he didn't have his own monitor.
[...] He somehow got to work. DOS 6.22 and all the device drivers “installed like a charm”. NetWare 4.1 installed. It was seen by both Windows 95 and 98 on the frail network. Now it came time to restore the application.
But it turned out that the stack of disks contained only data, not the application. Even the 10MB disk from the “old” server was uselessly corrupted.
Earl tried to explain this problem, but the client was having none of it and showed him the door.
Earl tells The Register the client owes him about US$5,000.00 for his time and is showing no signs of paying up. At least he didn't have to re-type all that source code: perhaps there weren't enough keyboards in the office!
(Score: 5, Interesting) by NCommander on Saturday August 06 2016, @01:14PM
As someone who has (and currently is) working full time freelancer, I can sympathize with Earl. Some clients are remarkably poor at providing people with what they need to successfully do the job. I had a more recent version of this involving development with VNC due to lack of hardware at previous employers.
Having a dot matrix printout of the source goes far better than how a lot of these stories go. I initially learned how to read x86 assembly for dealing with "ancient app with no source" usecase. The article doesn't say when this happened, but I'm guessing 2002-2003 back when doing a mass read-in OCR wasn't too common.
NetWare as a server side platform was a very very strange beast. In terms of programming, it was very much like DOS with Win 3.1 multitasking schematics. The entire thing ran in Ring 0, and NLMs basically linked right into the kernel. To get a working NetWare server, you had to link a special kernel with your drivers and application NLMs, reboot and pray. Relinking kernels isn't exactly unusual (it wasn't until Solaris 2.1 that kernel modules started becoming common in UNIX), but having to toss the application in was a bit far. Add in the fact the entire thing ran on IPX for networking until Netware 5, and you have an environment that is very unique.
Multitasking was cooperative in nature, and processes had to yield control back to the scheduler or the entire thing would go ka-flunk. NW ran with an identity memory map, so a buffer overrun could not only take out your app, it could go take out the kernel while it was at it. I've never heard of Pascal being used for app development on NetWare; as far as I know Watcom was the official compiler (NW support lives on in OpenWatcom), and that was about it (you could use GCC if you wanted to pull teeth).
Where Netware shined was it was stupid fast for its era, and since generally you had very few applications on it, it tended to be remarkably rock solid on the server despite the above (the client side was an entirely different baileywick). I never got a chance to play with it in production, but I did a fair bit of reading through the documentation and such. WOn't mind finding a few ISOs of older NetWare and seeing if I can coax it to run in a VM to study it.
Still always moving
(Score: 2) by Hyperturtle on Saturday August 06 2016, @05:20PM
I still have my Novell 3.12 server operational (although.. it's not particularly fast in modern terms). It is on a Cyrix 5x86 120mhz processor; about the speed of a pentium 90 or 100 in comparison. 4 IDE drives were used as the mountable file share.
For the network file share speed, it was definitely faster than NT 4.0 when it came to IPX (compared against both servers-- even though the NT 4.0 server eventually became a quad processor proliant). It took a gigabit card running TCP/IP to beat the Novell 100mb IPX to finally see a reason to migrate for my local storage needs. I still use it for a windows 98 laptop I have (which I recently loaded doom 1 and 2 onto for the fun of it and to try out Carmack's new e1m9 map prior to the latest Doom's release. Wireless IPX or go get fragged!)
In any event, I think a difference between you and me is that you are asking for ISOs of an older Novell Network Operating System, and I am envisioning reaching for a box of diskettes. I do not believe I actually ever owned a "Novell CD"; infact, none of the Novell servers I ever encountered even had a CD drive of any kind... Someone out there would have had the foresight to create what you are looking for while they still had a working floppy drive and working installation media.
I guess if you find that, let me know... I also would dig a copy of Lantastic or Banyan Vines while you are at it..
(Score: 3, Informative) by frojack on Saturday August 06 2016, @06:03PM
NetWare as a server side platform was a very very strange beast. In terms of programming, it was very much like DOS with Win 3.1 multitasking schematics. The entire thing ran in Ring 0, and NLMs basically linked right into the kernel. To get a working NetWare server, you had to link a special kernel with your drivers and application NLMs, reboot and pray.
It was never really intended for customer side processes running on the server. That all arrived late in its life cycle driven by third party backup tasks and database servers - all written by large companies partnering with Novell.
Nobody I ever ran into designed any customer applications to run on the server. Launched From the server, using data from the server, sure. But running ON the server, was avoided like the plague.
(I installed and maintained a lot of netware servers in my day).
Novell really didn't go out of their way to encourage server side applications.
And if you stayed away from crazy shit like that it was the most amazingly stable platform I've ever worked on. Many Virtual Machine environments still support Netware as a Guest VM.
No, you are mistaken. I've always had this sig.
(Score: 2) by NCommander on Saturday August 06 2016, @06:36PM
YOu know, thinking about this, I wonder of TFA is getting more than a few details wrong. An accounting app on NetWare does seem kinda ... odd, esp, considering its written in a language that isn't generally used for dev on NetWare.
Still always moving
(Score: 2) by archfeld on Saturday August 06 2016, @06:44PM
Reminds me of a story I read many years ago. Server 54 where are you ?
http://www.theregister.co.uk/2001/04/12/missing_novell_server_discovered_after/ [theregister.co.uk]
For the NSA : Explosives, guns, assassination, conspiracy, primers, detonators, initiators, main charge, nuclear charge
(Score: 1) by driven on Saturday August 06 2016, @01:15PM
For that amount of money, I'd be taking them to small claims court. Hopefully Earl has his case well documented.
(Score: 0) by Anonymous Coward on Saturday August 06 2016, @01:23PM
No, really, interesting one.
(Score: 2) by PizzaRollPlinkett on Saturday August 06 2016, @01:42PM
Ya think? A company which hasn't upgraded anything since the 1990s might be cheap? And they stiffed him the $5k?
Wow ... "a Pascal compiler could cross-compile for NetWare" ... surely he could get Turbo Pascal 7 off of a warez site or something? I mean, tools of that period aren't hard to find.
(E-mail me if you want a pizza roll!)
(Score: 3, Informative) by NCommander on Saturday August 06 2016, @01:43PM
NW was an unusual beast. Very little supports NLM, and I've never heard turbo pascal support it. I won't be surprised if the compiler was niehg impossible to find.
Still always moving
(Score: 3, Funny) by Anonymous Coward on Saturday August 06 2016, @04:27PM
A company which hasn't upgraded anything since the 1990s might be cheap? And they stiffed him the $5k?
I bet they'd pay him they could ... but their accounting system is currently down and they are waiting for a developer to fix it.
(Score: 0) by Anonymous Coward on Saturday August 06 2016, @02:03PM
I'm working a job now where I've put in a enormous amount of hours and at the end the problem is with the training of the personnel. They don't see it that way and I fear that they will not pay the bill. To make matters worse they still call in the old tech to 'fix' things and he is making a lot of noise about how I'm doing it all wrong.
(Score: 0) by Anonymous Coward on Saturday August 06 2016, @03:29PM
I used to work for a call center that regularly had customers default on $1k+ bills. I don't think there's a lot you can do other than sell it off to a collections agency. You do have everything you did, what you were asked to do, and the expectations you communicated to them documented and signed off, right?
Chances are, you're never going to see your full fee, say sorry. It sounds like you're caught in the middle of some kind of office drama going on. Somebody was upset with the old tech, so they called you in, except that the old tech had the job likely because of nepotism or just because life is unfair.
Disclaimer: I've never been in a position where I've needed to actually do collections. I just hear the same exasperated tale over and over again from accountants I've worked with.
(Score: 2) by rob_on_earth on Monday August 08 2016, @02:48PM
Document everything, dates times, meetings, people requests, everything, else its just your word against theirs.
Did a gig through an agency the client complained and said they would not pay. The agency got arsey until I showed them I had fulfilled all the requirements as per the original spec. I was not being pedantic, it was only a 3 day job but required me to fly to Ireland. Not getting paid was not an option.
(Score: 4, Funny) by Anonymous Coward on Saturday August 06 2016, @02:22PM
Welcome again to On-Call, our Friday wallow in jobs that are nastier than yours.
I once had a supervisor who informed me that creating shortcuts on my desktop was "destruction of government property", and threatened to have me refereed to the DOJ for investigation. Her office was next door to mine with a window partition. I would occasionally glimpse her typing to a blank screen during my sanctioned potty breaks.
I believe my soul consumed itself at some point during her tutelage.
(Score: 1, Funny) by Anonymous Coward on Saturday August 06 2016, @03:46PM
Heh. Call center AC here. One of the things I remember was that we weren't allowed to open notepad or calc even though those would have been incredibly helpful. Co-worker nearly got fired for using notepad to keep track of details about his caller's problem instead of using scrap paper.
I think I can picture your old boss exactly. Probably obese, right? Avid churchgoer? Babysits instead of manages?
(Score: 5, Interesting) by Marand on Saturday August 06 2016, @09:36PM
I once had a supervisor who informed me that creating shortcuts on my desktop was "destruction of government property", and threatened to have me refereed to the DOJ for investigation
Sounds like someone I worked for once. There was this Windows 95 system (well after it was out of support, mind; it was that kind of place) with so many files in the "My Documents" folder that you couldn't even look in it without the file manager exploding, and I got tasked with figuring out the problem. So I fixed it, moved the files (old reports, crap like that) into subdirs by date, showed what I did and where I put everything, and got a "good job, thanks" when everything worked properly again. Would have been a nice story if it had ended here, but no.
A day later, I got accused of deliberately breaking the computer and hiding files for "job security".
It turns out she only knew how to open files from the 'Recent Documents' menu and had no idea how to actually access files, so she had been opening the same ~8 year old doc from that menu every day for years. So, when the file got moved to a dated subdir, I got accused of deleting company data to deliberately interfere with her job. Despite showing where I put the files and how to access them when I did it, and showing it again after being accused of deleting it.
That wasn't even the worst thing I dealt with there, but it was the only one relevant to the topic. I got sick of everyone being used as a scapegoat for her incompetence (and her boss protecting her no matter what) and walked out on that job not long after.
(Score: 3, Interesting) by Anonymous Coward on Saturday August 06 2016, @10:46PM
There was someone like that at my old job. The worse part is that we all suspected (and I do mean "all," as most people who had to deal with her regardless of department) that some of that was actually purposeful "incompetence." My suspicion is that one day, one of the techs went full on BOFH on her and remoted in to her computer at night and changed a single formula in an Excel document in her MRU list. Took about a month for the problem to trickle up and get traced back to her. She couldn't handle the idea that she made a mistake and it blamed it everywhere. Her boss, trying to protect her dug through everything to blame someone else. Except, all the evidence pointed to her: the remote log was empty, the email archive of IT activity didn't mention it, the file access log had nothing, there were no "smoking-gun" emails, the changed numbers obviously came from her, and the numbers from Excel were put down on a paper report required for auditing. The higher-ups ended up getting rid of her and the boss for "wasting their time" with the witch hunt. The only reason I suspect sabotage is that the mistake didn't make sense for her to make, given what the content and purpose of the workbook was; not that I'm too broken up at the fallout.
(Score: 3, Interesting) by Marand on Sunday August 07 2016, @04:12AM
The worse part is that we all suspected (and I do mean "all," as most people who had to deal with her regardless of department) that some of that was actually purposeful "incompetence."
This part of your story sounds very much like the situation in my story. She'd be the one to set up schedules and then later "oh, I can't do that, I have a thing" and make someone else cover her ass. Constantly. To the point that she was probably not even working 20 hours a week. Acted like it was a mistake or something special came up every time, but bullshit on that.
And, despite shit like that, and her screwing up everything she touched (especially on a computer), her boss kept covering her ass. Did anything it took to deflect criticism. I went into overtime covering her work, and he asked me why. When I mentioned I hit it due to covering for her because she was constantly fucking off and skipping work, he told me I better find a way to do her job WITHOUT going into overtime next time and then quickly changed the subject.
It was like that constantly, not just for me but anybody else dealing with her. She was untouchable, because they were friends outside of work and had worked together for years. I heard things fell apart after I quit, because she still wasn't showing up for work half the time and nobody else would cover for her after I left. Yet she never got fired.
(Score: 4, Informative) by toygeek on Saturday August 06 2016, @02:34PM
I do computer repair on the side, have for a long time. For a while I trolled on Craigslist for jobs, but no more- the clientele were too far out for me. Case in point: Lady says her laptops are "hacked" and needs them fixed. She has two or three of them. I get to the location (her house) and the laptops are there, but the batteries are dead, and she can't find the power supplies. So I wait. She finally finds them and we get started. Then she tells me she has this desktop PC she wants me to set up for her, but again, she's lacking cables. Between caring for kittens, being mean to her kids, whatever else she was doing, she took forever to find anything. All told I was there for almost 3 hours and got the things she wanted done, but just barely. At the end of the service call she complained that I "took a really long time." Well no crap lady, you took an hour to find a cord.
There is no Sig. Okay, maybe a short one. http://miscdotgeek.com
(Score: 1, Interesting) by Anonymous Coward on Saturday August 06 2016, @07:53PM
Last two jobs were a Win 7 box somebody had legally or illegally upgraded to Win 7 Ultimate from Home Basic I think it was. Long story short it now defaulted to Aero, resulting in them thinking it had a virus. Switch Aero off, OMG it was so fast! Turned out it had an HD3000 IGP in it. Not very fast, as you can imagine. Additionally it was claimed to have reverted a week or two later, but I wasn't given a chance to verify if the issue was Aero again, or a virus that got missed while scanning (It was a particularly slow system, and since Aero appeared to fix the usability issues, I only finished a partial scan with the promise to run a full one if the usability issues returned, since they were in a hurry to get it back.)
The other bigger hassle was a favor for someone: It was a computer system for some sort of medical equipment that was running Linux. Turned out it used an array of IDE disks for the data storage, but a SCSI U320 for the OS. Having not futzed with SCSI since the early '00s I didn't have any of the required cables, and this idiot, despite being in the used computer resale market didn't have cables for it, nor supposedly IDE drives (he ended up finding one later.) It needed a password reset that required glitching the bios, installing a copy of linux on the donor drive, then repasswording the original SCSI drive (thanks to glitching the bios, which allowed using a non-default boot config, which it was secured against if a particular jumper wasn't shorted or open while the system was booting up. Literally, it would revert to defaults!) System got finished but the guy in question gipped me on the agreed upon amount of cash, and only paid the original amount, despite misrepresenting the job to me. The friend in question made good on his own payment, but I would never work with that other guy again (pretty sure he gipped my friend on his half of the sale too.)
Moral of my story? Two out of the last three jobs involved working with repairing software on legally questionable systems. It seems like about 3/4s of the independent tech jobs left entail that, and being smart enough to observe that makes it hard to feel comfortable doing side jobs like this given the potential legal burden you have from knowingly working on them. And how often does word of mouth references get you new work anymore, anyways? Maybe this experience is unique to me, however.
(Score: 0) by Anonymous Coward on Saturday August 06 2016, @08:20PM
Restoring a BIOS to defaults by changing a jumper is a feature that has existed almost since we've had BIOSes. What kind of tech support guy are you to not know that?
(Score: -1, Troll) by Anonymous Coward on Saturday August 06 2016, @08:46PM
You should have fucked the shit out of her.
(Score: 2, Informative) by Anonymous Coward on Saturday August 06 2016, @03:51PM
I was working for a <10 person consulting company, and a competitor's customer was expressing interest in switching to our product. But they wanted to see our product working on their server with their data first.
One of our developers went to their site to set up the system. He asked for a tape with which to make a backup of their entire system before making any changes, and they gave him an unmarked tape to do so. He made the backup, installed our software, and converted their data to our system.
While he was working, the system crashed and wouldn't come back up. Our post-mortem indicated that the memory was flaky (each time it booted, it would report differing amounts of installed memory), the hard drive was loaded with bad sectors, and the tape drive was writing garbage to the tapes.
We then discovered that they only had one backup tape, and it was the tape they provided to our developer to do his backup. It was unreadable, and the hard disk was almost unreadable. In particular, the customer's business data was gone.
Even though the system was badly broken before we arrived, since we were the last people to touch it, all of the problems were blamed on us (including "wiping out" their only backup tape). Our company ended up buying them a new server, and re-entering all of their business data from printouts they had laying around. The consulting company went bankrupt soon after.
I would like to think that I would have been more diligent if I had been doing the original work (I was dragged in after the sh*t his the fan to try to fix things), but other than verifying the backup tape before starting, our guy didn't do anything particularly wrong. (And, since the customer used their one and only backup tape as a scratch tape, if it had contained good data, it was ruined anyway).
At one point during our attempt to fix things, one of the customer's people laughed about how clever they were to blame us for their broken system, forcing us to give them a whole new system (and cleaning up their data in the process).
(Score: 3, Insightful) by Anonymous Coward on Saturday August 06 2016, @04:38PM
Our place has been bitten by that scam before. That is why we bring a new computer to show off our systems and copy data over. We also have sign offs and waivers and never do anything they should. Need to back up the system first? Ask one of them to do it and sign off that they did or that they accept all the risk of not doing so. Need data copied over? Give them the NFS share and have them do it or have them sign the waiver.
An additional benefit is that you can take the computer with you when you are done, keep root on it so they don't pirate, and, on one occasion, a potential client didn't buy but insisted that the disk stay (for "security purposes"), so my boss ripped it out while they were watching and took a drill to it and ripped off the controller board. The admin guy watching then turned his nose up after bitching that the disk was worthless now.
(Score: 1, Insightful) by Anonymous Coward on Saturday August 06 2016, @05:05PM
Indeed, and always always make sure there are no problems on the customer hardware prior to daring to touch it beyond more than log review.
They often won't let you just reboot servers to see if they come back up... but no one should have a problem letting you look at logs. If they don't know how to retrieve them, that is a red flag and has to be taken into context with the effort -- a system that ran itself is one thing, but a client seeking to avoid liability can forget how to do a whole lot.
(Score: 3, Informative) by sjames on Sunday August 07 2016, @08:28AM
First rule, if they want a demo on their system, they must sign off that it is a scratch system and may be wiped after the demo. They copy the data on to it so if they somehow screw it up, it's on them.
Software for the demo should include a logic bomb to wipe itself out.
(Score: 1) by purple_cobra on Monday August 08 2016, @11:41AM
A similar story convinced me to reconsider my career in IT. I can't remember half the details - it was 20 years ago and my memory was crap then - but it involved an ancient HP (IIRC) server running some wonky Unix-alike that we were required to shift from one workplace to another. We did the ritual 3x sync and shut it down, only to find the POS wouldn't come back up after we'd shifted it. Boss spent the whole weekend prodding it, which I'd have paid to see as he was clueless.
"Clueless" as in "I found this old light-up sign and plugged it in, causing the power to crap out. Can you get the Solaris box back up?". And I haven't mentioned the bloody idiot was too cheap to buy a UPS for this mission-critical box. Following that 3-4 hour reassembly/babysitting job, I got into work two days later to find he'd just plugged the same sign into the same power loop causing the same result. He did finally buy a UPS after that, but I left not too long after. Think the company waddled along for another 8 months or so before all the customers ran off screaming.
(Score: 3, Insightful) by CirclesInSand on Saturday August 06 2016, @05:03PM
Earl tells The Register the client owes him about US$5,000.00 for his time and is showing no signs of paying up.
That's what contracts are for. If you had one, you wouldn't need to tell the register. Always always always get a contract.
(Score: 2) by NotSanguine on Saturday August 06 2016, @06:34PM
The developer, it turned out, had died in 2001. But the source code was in the company safe … on about 2000 pages of dot matrix printer paper.
At least they printed out the source code rather than the binary executables. Don't laugh, it's been done.
No, no, you're not thinking; you're just being logical. --Niels Bohr
(Score: 0) by Anonymous Coward on Saturday August 06 2016, @09:44PM
While "Earl" probably isn't going back, if he did he might be able to scan and OCR that dot-matrix printed source code. A whole lot easier than re-typing it in (assuming he can tweak the OCR software to do a good job).
(Score: 2) by istartedi on Saturday August 06 2016, @11:48PM
As bad as printing out the executable is, at least it's something. I typed in
some executables for the C64 from the old Compute! magazine, with a checksum
on each line and all that. It really wasn't that bad, and they worked. Of course today's
code is so much larger it's not practical. Maybe you could scan it in, but yeah really...
just burn the source to a high quality CD and it should last a long time. I'd say thumb
drive, but I've actually had one go bad whereas I've got old Kodak gold CDs from the 90s
that probably still work. I know I was able to read them after 10 years. It's been a long
time since I've dug them out...
Appended to the end of comments you post. Max: 120 chars.
(Score: 2) by cellocgw on Sunday August 07 2016, @12:35AM
Gotta warn you, Alex, Soylent News is trying to horn in on your turf here. Better fight back with lawyers, or something worse.
Physicist, cellist, former OTTer (1190) resume: https://app.box.com/witthoftresume