Arthur T Knackerbracket has processed the following story:
In the old days, if you wanted to snoop on a piece of serial gear, you probably had a serial monitor or, perhaps, an attachment for your scope or logic analyzer. Today, you can get cheap logic analyzers that can do the job, but what if you want a software-only solution? Recently, I needed to do a little debugging on a USB serial port and, of course, there isn’t really anywhere to easily tie in a monitor or a logic analyzer. So I started looking for an alternate solution.
If you recall, in a previous Linux Fu we talked about pseudoterminals which look like serial ports but actually talk to a piece of software. That might make you think: why not put a piece of monitor software between the serial port and a pty? Why not, indeed? That’s such a good idea that it has already been done. When it works, it works well. The only issue is, of course, that it doesn’t always work.
The software in question is interceptty. You may have to build it from source, but there aren’t any oddball dependencies. [...]
[...] The software uses the concept of a backend device and a frontend device. The back device is, usually, your normal serial port. The frontend device is something that interceptty creates. So the idea is that you connect the program to the backend, it creates the front end, and then you connect some other program to the front end. The program will log all the traffic between the program connected to the front end and the port on the back end.
You can also use file descriptors, unix sockets, or TCP sockets as a front or back device. The backend can also be a running program. There is also a provision for connecting between two actual ports. You can find all the options on the program’s man page.
The output is a bit awkward, but it is easy to parse by other programs including an example Perl program included with it. A character shows you the direction of the data then you see a character in both hex and ASCII.
[...] Unfortunately, not all software likes to work with ptys. In particular, the main program I wanted to use takes advantage of the Sigrok serial port library. It is a known issue that this library makes calls that don’t work well with ptys. However, if you use just about any normal terminal program like picocom or tio, it works fine. Other serial libraries seem to be able to handle it, also. I thought about trying slsnif, but it works the same way, so I doubt it would be any better in that regard.
Have any of our community got alternative solutions to this problem? [JR]
Related Stories
Hardware designer and manufacturer, SparkFun, has a short biography about computer engineer Ajay Bhatt who is widely recognized as one of the key inventors of the Universal Serial Bus (USB).
Once the design was finalized, Bhatt and his team worked with other technology companies to promote and standardize the USB. They formed a working group called the USB Implementers Forum (USB-IF) to develop the USB specification, which was first introduced in 1996.
The USB specification quickly gained widespread adoption in the technology industry due to its convenience and versatility, and new versions of the standard were introduced over the years to improve data transfer speeds, power management, and other features. Today, the USB is used in a wide range of devices, and it continues to evolve and improve with each new iteration.
When Intel initially developed the USB, it held the patents for the technology, which allowed the company to control the standard and charge licensing fees for its use. However, Intel soon realized that its proprietary approach was not in the best interests of the industry or consumers. The company recognized that the success of the USB depended on its widespread adoption and interoperability with different devices, which would not be possible if licensing fees were required for every use.
In response, Intel took a bold step and transferred ownership of the USB specifications to a non-profit organization called the USB Implementers Forum (USB-IF). The USB-IF is a group of companies that work together to promote and develop the USB standard, with the goal of ensuring that the standard remains open and accessible to all.
Intel's decision to transfer ownership of the USB specifications to the USB-IF was a pivotal moment in the development of the USB standard. It helped to ensure that the USB became a truly universal and open interface, which has had a profound impact on the computer industry and consumers around the world. Today, the USB is used in a wide range of devices, from computers and smartphones to home appliances and automotive systems, and it continues to evolve and improve to meet the needs of an ever-changing technological landscape.
Previously:
(2022) Henn Tan and the Invention of the USB Thumb Drive in Singapore
(2022) Linux Fu: Eavesdropping On Serial
(Score: 4, Interesting) by Rich on Monday September 12 2022, @10:41AM (3 children)
I've dealt with ptys a bit in the past. I worked on a device that had about a dozen robotic subsystems that were all serially connected. The customer had mainboards with a heapload of serial interfaces, originally 68000 with SC28L198 (?) octal UART, then PPC with MAX3110 SPI-UARTs. (I wrote the Linux drivers for the MAX3110 and found them to be awfully buggy, losing interrupts and needing timer polling to clean up...).
Anyway, I put all this into a simulation (full 3D hardware with OpenGL visuals, including fluid piping, sensing, and reacting into different properties) and had to access the now virtual robot nodes. I used ptys for this and found them rather awkward. There were the statically allocated ones, which have no proper arbitration, and the dynamic ones which could only be accessed through semi-stable links created after the ptys. I wish there would have been something like "mkpty", like there is "mkfifo".
While this doesn't help with the call for hints, it points out the overall awkwardness. But maybe some future OS designer reads these lines and considers a clean userspace device relay, so that one can "mknod" and open that from both sides, where a daemon sits at the back and can appropriately handle not only the data, but all the ioctl, too. I probably just described a microkernel. (For my example, I could have done one of those daemons, with a pty on either side, and forwarding, but iirc for simplicity I just settled on a defined starting order, e.g. the virtual device had to set up its links, and only then could the controlling software be launched).
(Score: 0) by Anonymous Coward on Monday September 12 2022, @11:26AM (2 children)
I haven't had to deal with that kind of thing for years, but it amazes me that for as long as ptys have been around, that the issues you mention weren't addressed.
(Score: 4, Interesting) by ls671 on Monday September 12 2022, @12:49PM
It has always been addressed with "proxies" which is exactly what the article suggest so nothing really new here.
I built my own proxies for all kind of stuff. On the early PC, you could catch the keyboard interrupt, log any thing you wanted or take specific action then relay the key pressed to the program by calling the original interrupt code. The list is almost endless where using some kind of proxy in between allows you to easily debug.
Everything I write is lies, read between the lines.
(Score: 3, Interesting) by Rich on Monday September 12 2022, @04:26PM
Well, once something is in place in such a (now) conservative environment as UNIX, it stays. The "mknod" filesystem entries have a major and minor number, and that's used to link them up to drivers, and everything works from there. IIRC, last time I did a small (uclibc/busybox) embedded system with static devices, I still had to install the nodes with correct numbers manually. For interactive use, devfs was a great step into the right direction, but it's all a bit awkward how that needs to be processed for presentation by some userland agent (udev, devicekit, or whatever is en vogue today, and not that surfsticks that pretend to be a block device and have to be convinced that they are Hayes modems to become a PPP network interface help with that).
What's also legacy is the whole line discipline stuff of the Linux serial ports, with all the cruft needed to support serial terminals baked into the drivers ('man stty' for the full horror story). It's all there for a reason, like when you want to control your machine from an unmodified VT52 or through a Hayes AT-compatible modem. ('getty' and its descendants).
(Score: 2) by drussell on Monday September 12 2022, @03:41PM (1 child)
On FreeBSD, I've always been able to just use the snp(4) [freebsd.org] device...
According to the manpage [freebsd.org], it's been included since FreeBSD 2.1, so I guess I couldn't have used it back on my original 1.1, 1.5 and 1.5.1 installs, but I wasn't running banks of modems and needing to mess with watching serial ports really until the days of 2.x, so I probably never needed it that far back. I haven't used it in many, many years, but I assume it still just works.
(Score: 2) by drussell on Monday September 12 2022, @03:49PM
Oh, the included utility to use the snp(4) device is called watch(8) [freebsd.org], by the way.
(Score: 3, Interesting) by VLM on Monday September 12 2022, @04:41PM
The problem in the article can be understood as some microcontrollers just have some wires into the chip and the USB lives entirely in the chip, there's no "usb to serial" converter chip like low end stuff, like an arduino or whatever.
This is classic FOSS because if you're paying an engineer $200K/yr at work, you pay like 1/20th that for a COTS USB protocol analyzer (not a protocol analyzer that connects over USB, but a protocol analyzer for USB protocol) and call it good, and it works great the first time with customer support if necessary and everything. Actually $10K is barely entry level but it'll work. Anyway the point is for a day job there's hardware that'll do this, and a lot more, really cheap compared to labor costs of not buying/renting the gear.
Like a now discontinued Keysight 4611. That thing was cool but a PITA. If I was the one doing the paying I'd get the competing LeCroy product for only mid four figures but whatever.
Nobody buys enough of this weird stuff for it to show up on ebay. If you want an oscope capable of doing competitive 1995 analog/RF work, you buy a 1995 scope off ebay for nothing. Also you probably aren't capable of working to the limit of 1995 analog/RF technology anyway, so buy something even cheaper, some Tektronix wonderscope from the 60s-70s. That was good enough to land people on the moon so you can probably use it 2022 to figure out why an arduino dont blink no more.
So its only fixable at home doing FOSS where the budget is "whatever the wife won't notice me buying"
(Score: 3, Interesting) by rleigh on Monday September 12 2022, @07:18PM (1 child)
Writing such a PTY passthrough was the first thing I wrote for testing PTYs and TTYs, using an intermediary process to shunt the data back and forth. My first multithreaded C++ application, in fact. I used blocking I/O in each thread for RX or TX to simulate the same blocking behaviour on the proxy TTY as the real one, but today I'd just use poll(2) if I was to repeat it. It works, but as others have said, Linux (and Unix) terminals and line disciplines are somewhat archaic, and the termios and stty stuff can't all be easily proxied/virtualised. You'll probably want to keep the TTY raw.
If you have the means to solder three wires on (GND, RxD, TxD), just hook it straight up to a logic analyser or a mixed-signal scope. Or do the same on a USB passthrough for (GND, D+, D-) and decode the USB CDC-ACM packets. I have a Saleae for this, which is super nice, but even the cheapest $15 ones are going to be perfectly fine for decoding basic serial data. They are the right tool for the job.
If Linux had ever got SVr4 STREAMS support, you could have just added in this capability via a custom STREAMS driver. Unfortunately, the Linux serial/tty implementation is one of the dirtiest and least featureful parts of the kernel. It's not even on par with SVr4 of 30 years back, which is a little bit tragic.
Another approach would be to bypass the PTYs and modify the Linux USB-serial driver or tty driver and have that dump out the data for you. Maybe with a couple of character devices to give you the raw RX and TX data streams. We had (sort of) monitoring capabilities for the virtual consoles with the vcs and vcsa devices which allowed the terminal device contents to be dumped, but we never got the same capabilities for serial datastreams. Not sure how hard that would be to add, but might be a fun little project to hack it in.
(Score: 2) by janrinok on Monday September 12 2022, @07:33PM
We know that it is easier using hardware - but TFS actually says "but what if you want a software-only solution?". This was looking at a purely software way of solving the problem.
But an interesting comment nevertheless.
(Score: 2) by Rich on Tuesday September 13 2022, @12:15AM
See comment https://sigrok.org/bugzilla/show_bug.cgi?id=1642 [sigrok.org] for the background
and pull request https://github.com/sigrokproject/libserialport/pull/4 [github.com] for a fix suggestion.
The maintainer doesn't seem to like the idea, but it should be possible to build a libserialport to get past the specific monitoring problem at hand in a somewhat dirty fashion.