Stories
Slash Boxes
Comments

SoylentNews is people

posted by NCommander on Wednesday March 15 2017, @04:00PM   Printer-friendly
from the I-will-show-myself-out-for-that-pun dept.

Last time — with the help of the excellent Michal Necasek of the OS/2 Museum — we talked about mapping the damage within the existing Xenix 386 disks and successfully got the system to the end of installation.

*

For those new to the series, I recommend you catch up with the previous three articles:

  1. Restoring Xenix 386 2.2.3c, Part 1
  2. Xenix 2.2.3c Restoration: No Tools, No Problem (Part 2)
  3. Xenix 2.2.3c Restoration: Damage Mapping (Part 3)

Unfortunately, at this point we had exhausted the data we could successfully recover from the TeleDisk images, so now it was time to think laterally in our quest to restore viable installation media. Back in Part 2, I posted the disk headers from each disk indicating what it was and where it was in the set:

./tmp/_lbl/prd=xos/typ=386AT/rel=2.2.3c/vol=N03
./tmp/_lbl/prd=xos/typ=n86/rel=2.2.2c/vol=B02
./tmp/_lbl/prd=xos/typ=n86/rel=2.2.2c/vol=X01

And also noted that there was a slight version mismatch (2.2.3 vs 2.2.2). What I didn’t point out was the type was different: n86, vs 386AT; Xenix speak for “generic x86” vs. “386 AT”. As Michal and I discussed it, I realized there was another place we could go to find sectors.

In-Depth Analysis

By extracting what I could from the damaged archives, what struck me the most where the modification times of some of the binaries:

file modification dates

April 13th, 1987. Quite a bit before the files on the N disks, which date to late November 1987 to early 1988. The file utility (which understands x.out headers) also showed that these binaries were *not* 386 specific:

$ file *
capinfo: ASCII text
fixpad:  Microsoft a.out [..] V2.3 V3.0 86 small model executable
mapchan: Microsoft a.out [..] V2.3 V3.0 86 small model executable
tic:     Microsoft a.out [..] V2.3 V3.0 86 small model executable
tid:     Microsoft a.out [..] V2.3 V3.0 86 small model executable
tput:    Microsoft a.out [..] V2.3 V3.0 86 small model executable
trchan:  Microsoft a.out [..] V2.3 V3.0 86 small model executable
uupick:  ASCII text
uuto:    ASCII text

Given we had already struck paydirt with the International Supplement and /etc/init, was it possible that (nearly) identical binaries might have shipped as part of another Xenix release?

At the time, SCO had three known releases of Xenix 2.2 for the x86 architecture: a 286 version, a 386 version for AT-compatibles, and a 386 version for PS/2. At the time, only the first one was known to exist, and more importantly, had been dumped. As such, I grabbed a copy of Xenix 286 2.2.1, the closest version known to match what we had.

Unlike the 386 version, Xenix 286 was shipped in the form of 7 high-density 5.25-inch floppy disks (1.2 MiB) instead of low density disks. After extracting the archives, I found exactly what I was looking for:

./tmp/_lbl/prd=xos/typ=286AT/rel=2.2.1e/vol=N02
./tmp/_lbl/prd=xos/typ=n86/rel=2.2.1c/vol=B01
./tmp/_lbl/prd=xos/typ=n86/rel=2.2.1c/vol=X01
./tmp/_lbl/prd=xos/typ=n86/rel=2.2.1c/vol=X02
./tmp/_lbl/prd=xos/typ=n86/rel=2.2.1c/vol=X03

Not a perfect match, but the Extended Utilities disks were de-facto shared between the 286 and 386 versions. Running cmp against a few binaries revealed that the vast majority were identical, though a few were different. This showed that SCO only recompiled something if it changed vs. doing a blanket recompile which likely was done to help reduce QA workload. Jackpot!

By carefully comparing with a hex editor to make sure the ends matched, I transplanted the missing sectors from the donor, and put them into the correct places in the X disks. That got me a reassembled X1, one of the two sectors in X3, and X4. Unfortunately, doscat on X2 had changed, so I set that aside for now. Nonetheless, halfway there.

Tar Format Details

Having hit paydirt, I turned my attention to the parts of the disk where a missing sector had bisected a file on X3, and X5. As technology has moved on over the years, even time-tested utilities such as tar have seen changes to keep up with the times. The archives on the disks were in what was known as the original “v7” tar format which has a simple and straightforward header (from Wikipedia):

Offset Size Field
0 100 File name
100 8 File mode
108 8 Owner's numeric user ID
116 8 Group's numeric user ID
124 12 File size in bytes (octal base)
136 12 Last modification time in numeric Unix time format (octal)
148 8 Checksum for header record
156 1 Link indicator (file type)
157 100 Name of linked file

Additionally, tar works on the concept of blocks, where are 512 bytes in length. My examination of the installer revealed that the disks were written with a blocking factor of 18, or in other words, each file was chunked into records of 9216 bytes (512*18). If the file wasn’t an exact multiple of 9216, it would be padded out to that length, and then the next record header would follow.

The tar header itself is also stored in its own unique block, and padded out to 512 bytes. For those keeping track at home, a missing sector on these disks were also 512 bytes. You might see where I’m going with this. By sheer luck, all three bifurcations happened in this padding section, and had annihilated the header, but left the binary data following it complete intact. By manually extracting this data, I confirmed that the binaries in all three cases matched binaries on Xenix 286.

By working backwards from the start of the binary data, I located the exact place in the archive the header *should* be. Due to the way blocking worked, the header would always be at the start of the 512 byte boundary that had been annihilated in the bad dumps. Since I knew the binaries matched the older release, I simply dropped in the old tar headers, saved the files, and “blamo”. That restored everything expect doscat, and a test on the VM confirmed I got working (and valid!) binaries. X1, and X3-5 were now fully restored.

Using the same technique, Michal reassembled the games disk.

Interlude: Serial Garbage

Before we get into talking about doscat, one thing that had been driving me up a wall was I couldn’t get the serial ports to work. They were being properly enumerated, and I could even bring them up with “enable tty1a”, but shortly afterwards, I’d get the following message:

“garbage or loose cable on serial dev 0, port shut down”

Searching on USENET showed that this was a relatively common problem in the era, and SCO even released a SLS update for Xenix 2.2 286 and 2.3 386 for it. Unfortunately, they *didn’t* release the update for 2.2 386, and even with that update, it appears this bug may have persisted well into the future as I could find reports of UnixWare reporting this error under VMWare.

I managed to identify the serial interrupt vector (_sioinir), but before I could even dig deeply into it, Michal beat me to the fix. I’ll let him explain in his own words:

The problem is that the emulated serial port is “too fast” and outgoing data may move over the virtual wire as fast as it is written. The Xenix driver does not expect that and if it successfully writes 10 characters in a row from its interrupt handler, or more accurately if there are still pending interrupts after ten loops through the interrupt handler, it throws up its hand and goes off to sulk in the corner.

Fortunately it’s easy to patch out the 10-loop limit and get functioning serial terminals.

(in this specific case, NOPing out the inc instruction in the serial driver).

With an even more patched kernel in place, I could now raise a terminal and enjoy copy and paste!. Unfortunately, vi wasn’t completely happy with this; Xenix doesn’t understand xterm termtype, and it *really* wants a legitimate vt100 on the other end of the connection. Still for basic data copying, it was wonderful, and would open the door to doing MicNet, and UUCP in the future.

As of this writing, I haven’t taught Xenix what an “xterm” is, but I could probably add it to it’s termcap database and rebuild it for a fully functional vi. Interesting, DTTerm from CDE works perfectly, and vi is much happier with it than the standard gnome-terminal or PuTTY.

And with that side trip over, let’s get back to doscat.

doscat

doscat was the first sector where reconstruction vs. recovery came into play. As I mentioned before, Xenix used its own variation of the x.out binary format to support multiple code and data segments. In this specific case, the missing piece of data was located towards the tail end of the binary — in the data segment — not far past the end of the string table. Given that we had two other versions of doscat to compare it to, Michal and I tried to reconstruct it.

F6s in doscat

Our one hint was the missing data block started with 02. Fortunately, a known copy of the Xenix development tools for 386 2.3 has survived, and better still, it was install-able with the "custom" utility.

Insert D1 Manipulate packages screen Select packages feeding disks

The ‘hdr’ utility confirmed that the missing bit was indeed in the data segment:

# hdr /usr/bin/doscat
magic number:   206     (x.out)
ext size:       002c
text size:      00004141
data size:      00000bbc
bss size:       00000f64
symbol size:    00000000
reloc table:    00000000
entry point:    003f:0000
little endian byte ordering
little endian word ordering
cpu type:       8086
run-time environment:
        Xenix version 5
        segmented format
        Small model, fixed stack, pure text, separate I & D, executable
stack size:     00001500
segpos:         00000060
segsize:        00000040

Using adb to explore the broken data segment, with the default fill pattern of F6, it would segfault with trying to write to binary location 0xF6F6, but doscat -h worked. Nulling out the block caused all output from the command to die. Michal managed to determine it was part of the __iob block which defines standard I/O operations, and successfully copied it from another version of doscat.

I think the area just before the end of the missing sector within ‘doscat’ contains the __iob array or something like that. The 02h byte which survived was almost certainly supposed to be preceded by 06h and a few other bytes. If it’s all zeroed, there will be no I/O (stdout closed or whatever). If F6h bytes are in place, some I/O might happen.

The 02h likely corresponds to the _file field of the stderr entry.

My earlier attempts to reconstruct the sector were lead with failure as I had a similar (but incorrect) block of data. With the rebuilt sector in place, I could successfully install all the Extended Disks, and have a full set of working utilities. Unfortunately, due to the hosed manifest file, /etc/custom which is used to install addon software still wasn't functioning. However, manual installation via tar -xf /dev/fd0 was perfectly viable. With all the extended utilities in place, the system was considerably more complete. One step closer to full restoration. All that remained at this point were four sectors: N1’s manifest, N4’s libmdep.a, N5's adb, and I3’s installation script.

Scoreboard

As I mentioned in the previous article, a copy of Xenix 386PS emerged the day before I posted part 1 of this series. As suspected, the version of the Extended Utilities it shipped was identical in version we were rebuilding, so we could actually see how close we were. When Michal compared them side-by-side, he found that X1, 3, 4, and 5 were identical to our reconstructions!

Talk about hitting a home run! X2 was different since the rebuilt doscat sector was something of a guess of how it should work. Still, we always knew that at best this was going to be a reasonable approximation of what SHOULD be on the disks. A functional reconstruction is better than no reconstruction after all. As part of this work, we also learned a lot about the SCO copy protection which became a pain in the butt for rebuilding N4.

The Road Ahead

Looking ahead, I’ve likely got one or two more articles on reconstruction, some interesting bits of history we gleaned from an in-depth study of the Xenix compiler (which has a surprising ancestor), an article demoing some Xenix apps such as Microplan, MicNet, and if I can set it up properly, UUCP support (plus discussing bang-path addressing). After we finish with that, I might have some interesting material on NeXTstep to write up, and perhaps reboot my old Itanium box and get some OpenVMS coverage.

On the whole, I think the reception to these type of articles has been good, and I’d like to thank our subscribers for their support in helping keeping SoylentNews up for three years now. As we proceed into 2017, I’m hoping we’ll reach a level of success where we can repay the stakeholders, and perhaps even have a budget for obtaining and exploring even more interesting pieces of technology.

~ NCommander

Related Stories

Restoring Xenix 386 2.2.3c, Part 1 65 comments

One of my favorite hobbies is both retrocomputing projects, and the slightly more arcane field of software archeology; the process of restoring and understanding lost or damaged pieces of history. Quite a while ago, I came across the excellent OS/2 Museum, run by Michal Necasek which helps categorize many of the more obscure bits of PC history, including a series of articles about Xenix, Microsoft’s version of SVR2 UNIX.

What caught my attention were two articles talking about Xenix 386 2.2.3c, a virtually undocumented release that isn’t mentioned in much if any of the Santa Cruz Operation's (SCO, but see footnote) surviving literature of the time. Michal documented [1], [2] his efforts at the time, and ultimately concluded that the media was badly corrupted. Not knowing when to give up, I decided to give it a try and see if anything could be salvaged. As of this writing, and working with Michal, we’ve managed to achieve approximately 98% restoration of the product as it would have existed at the time.

Xenix 386 booted with uname

I’m going to write up the rather long and interesting quest of rebuilding this piece of history. I apologize in advance about the images in this article, but we only recently got serial functionality working again, and even then, early boot and installation has to be done over the console.

* - SCO in this case refers to the original Santa Cruz Operation, and not the later SCO Group who bought the name and started the SCO/Linux lawsuits.

Read more past the fold.

Xenix 2.2.3c Restoration: No Tools, No Problem (Part 2) 38 comments

When we last left off, with the help of the excellent Michal Necasek of the OS/2 Museum, we had gotten the damaged Xenix 2.2.3c past the first hurdle of installation, and directly into a post-reboot crash, the cause of which (at the time) I suspected was another emulation failure.

*

Needless to say, I needed to get past this. At this point, I have been examining the raw images as best I can, and figuring out how the installer comes together. After a few experiments, I managed to determine a few basic facts about how Xenix is installed when booting from N1/N2:

  • Coming out of reset, the Xenix kernel loads from N1, prompts for the "filesystem floppy" and starts /etc/init or /etc/inir coming out of IPL
    • Init prints the "Entering System Maintenance Mode" line.
    • Inir is used for running fsck, if necessary. Afterwards, it starts init.
  • Initially, the system starts /usr/lib/mkdev/hd which does the following
    • Format the Xenix partition, create slices, and mount the hard drive under /mnt
    • Compile the keyboard configuration files so the language selection sticks past reboot
    • Create a few device nodes manually
    • Cpio is used to copy a list of files from N2 to the root partition.
    • /profile.hd is copied to /mnt/.profile
    • Install a boot sector to the MBR of the hard drive and configure the bootloader for HDD booting
    • The hd script does the initial language selection, followed by fdisk
  • After the completion of the script /.profile is run as the system tries to bring up the root prompt which reboots the system after prompting that N1 should be re-inserted after the boot prompt.

So with knowing what the installer is trying to do, it was time to try and get down and dirty with it.

Xenix 2.2.3c Restoration: Damage Mapping (Part 3) 32 comments

This is the third exciting installment of this ongoing saga of restoring Xenix 2.2.3c. When we last left off, we had discovered that it is possible for Xenix to boot with several sectors missing in /etc/init and that the vast majority of the data on the diskettes was intact — to the point that several more steps of the early installation completed.

post reboot, insert disk

The story thus far:

[Aside: Quite a while ago, I came across the excellent OS/2 Museum, run by Michal Necasek which helps categorize many of the more obscure bits of PC history, including a series of articles about Xenix, Microsoft’s version of SVR2 UNIX.]

If we were to get any further, Michal and I would have to dig deep into the world of teledisk, floppy disk formats, and perform some creative thinking.

This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough

Mark All as Read

Mark All as Unread

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
  • (Score: 5, Insightful) by epitaxial on Wednesday March 15 2017, @08:09PM

    by epitaxial (3165) on Wednesday March 15 2017, @08:09PM (#479556)

    I really enjoyed this series of articles. I had read about Xenix but never encountered it anywhere in the wild. Fascinating to see how easily magnetic bits can be lost.

  • (Score: 3, Interesting) by Valkor on Wednesday March 15 2017, @08:48PM (2 children)

    by Valkor (4253) Subscriber Badge on Wednesday March 15 2017, @08:48PM (#479572) Homepage

    There are images in the article! Heresy!

    Me and some friends set up a Xenix vm a while ago. Since we couldn't find a working IP stack for it, we interfaced using a telnet serial gateway. It was tons of fun because only one person could use it at a time, but any number of folks could connect and interfere with it. Good to see you got things going!

    • (Score: 1, Informative) by Anonymous Coward on Thursday March 16 2017, @03:53AM (1 child)

      by Anonymous Coward on Thursday March 16 2017, @03:53AM (#479667)

      Yeah. This is what I was wondering.
      Once the OS is running, what about apps?

      MSFT's Xenix: A minuscule temporal window

      https://en.wikipedia.org/wiki/Xenix [wikipedia.org]
      Microsoft [...] purchased a license for Version 7 Unix from AT&T in 1978, and announced on August 25, 1980, that it would make it available for the 16-bit microcomputer market. Because Microsoft was not able to license the "UNIX" name itself, the company gave it an original name.

      Microsoft, in turn, did not sell Xenix directly to end users; instead, they licensed it to OEMs such as IBM, Intel, Management Systems Development, Tandy, Altos, SCO, and Siemens (SINIX) who then ported it to their own proprietary computer architectures.

      [...]the first [shipment to a customer of a device running Xenix] was January 1981
      [...]
      Microsoft [...] planned to over time improve MS-DOS so it would be almost indistinguishable from single-user Xenix
      [...]
      AT&T started selling System V, however, after the breakup of the Bell System [in January 1982]. Microsoft, believing that it could not compete with Unix's developer, decided to abandon Xenix.

      -- OriginalOwner_ [soylentnews.org]

      • (Score: 2) by NCommander on Thursday March 16 2017, @05:45AM

        by NCommander (2) Subscriber Badge <mcasadevall@soylentnews.org> on Thursday March 16 2017, @05:45AM (#479679) Homepage Journal

        I have a set of Xenix software to demo which will go at the end of this series such as Microplan, and possibly MGR if I can get it to build. There was also Word for Xenix and SCO UNIX but that seems to have disappeared off ye olde internet. Only newer version for SCO UNIX.

        --
        Still always moving
  • (Score: 1) by Ethanol-fueled on Wednesday March 15 2017, @11:40PM (2 children)

    by Ethanol-fueled (2792) Subscriber Badge on Wednesday March 15 2017, @11:40PM (#479610) Homepage Journal

    I missed the previous versions and looked at them, and I'm not going to pretend I know shit about computing but I found the language menu:

    1. American
    2. British
    [...]

    Pretty hilarious. If only Stormtroopers of Death [bestrocktshirts.com] released an album called "Speak American or Die."

    • (Score: 2, Touché) by Anonymous Coward on Thursday March 16 2017, @01:03AM

      by Anonymous Coward on Thursday March 16 2017, @01:03AM (#479623)

      Why is it hilarious?

      Brits speak English.

      Americans mumble and slur their way through a loosely related dialect.

    • (Score: 2) by NCommander on Thursday March 16 2017, @07:42AM

      by NCommander (2) Subscriber Badge <mcasadevall@soylentnews.org> on Thursday March 16 2017, @07:42AM (#479693) Homepage Journal

      That's actually keyboard selection, not language support. The base system uses American English, but the international suppliment includes 8-bit clean binaries which could theorically also be translated. I speak relatively decent Spanish, but I haven't actually tried it because a lot of those packs like to remap the keyboard and I've had a bad track record with that and virtualization.

      --
      Still always moving
  • (Score: 3, Interesting) by evilviper on Thursday March 16 2017, @02:53AM (1 child)

    by evilviper (1760) on Thursday March 16 2017, @02:53AM (#479651) Homepage Journal

    Xenix doesn’t understand xterm termtype, and it *really* wants a legitimate vt100 on the other end of the connection.

    I feel your pain, there. What terminal types are available? Wyse was very popular with SCO. If Xenix has wy50 or wy60 in there, you have good options for proper emulation. See:

    https://www.freshports.org/comms/wy60/ [freshports.org]

    --
    Hydrogen cyanide is a delicious and necessary part of the human diet.
    • (Score: 2) by NCommander on Thursday March 16 2017, @05:44AM

      by NCommander (2) Subscriber Badge <mcasadevall@soylentnews.org> on Thursday March 16 2017, @05:44AM (#479678) Homepage Journal

      wy50 and 60 were on the international suppliment disks, which I got working and installed. It more or less works properly as long as I use CDE's dterm vs. gnome-terminal/xterm so it's sorta moot right now, but I'll keep that in mind the next time I run into this specific issue. Having usable vi is nice.

      Right now, I've hit an issue with VirtualBox and serial ports that I can only seem to connect to named pipes once then VB goes AHAHAHAH fuck you and breaks it until I restart socat. Might be an issue with Xenix, or VirtualBox but it's hamstrung any efforts to make UUCP go.

      --
      Still always moving
  • (Score: 4, Informative) by dry on Thursday March 16 2017, @03:13AM

    by dry (223) on Thursday March 16 2017, @03:13AM (#479658)

    I see action on Michal's RSS feed and sure enough, 3 stories about Xenix. http://www.os2museum.com/wp/ [os2museum.com]

  • (Score: 3, Funny) by iWantToKeepAnon on Friday March 17 2017, @03:45PM

    by iWantToKeepAnon (686) on Friday March 17 2017, @03:45PM (#480460) Homepage Journal
    I know slashcode was old, but I don't think it will run on Xenix 2.2.3c. :-p
    --
    "Happy families are all alike; every unhappy family is unhappy in its own way." -- Anna Karenina by Leo Tolstoy
(1)