Stories
Slash Boxes
Comments

SoylentNews is people

posted by janrinok on Saturday April 20 2019, @11:10AM   Printer-friendly
from the schschschschschschschsch dept.

Famed hardware hacker Bunnie Huang has posted an overview of his notes on designing an open source entropy generator. His summary links to the full notes which include schematics, measurement results, as well as other key details.

The final optimized design takes <1cm2 area and draws 520uA at 3.3V when active and 12uA in standby (mostly 1.8V LDO leakage for the output stage, included in the measurement but normally provided by the system), and it passes preliminary functional tests from 2.8-4.4V and 0-80C. The output levels target a 0-1V swing, meant to be sampled using an on-chip ADC from a companion MCU, but one could add a comparator and turn it into a digital-compatible bitstream I suppose. I opted to use an actual diode instead of a NPN B-E junction, because the noise quality is empirically better and anecdotes on the Internet claim the NPN B-E junctions fail over time when operated as noise sources. I'll probably go through another iteration of tweaking before final integration, but afaik this is the smallest, lowest power open-source avalanche noise generator to date (slightly smaller than this one [PDF]).


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 3, Informative) by Rich on Sunday April 21 2019, @02:01PM (10 children)

    by Rich (945) on Sunday April 21 2019, @02:01PM (#832938) Journal

    The Horowitz/Hill circuit is the BE-junction of a 2N4401 through a 24dB/oct HP filter with the middle poles also being fixed gain stages. The output runs into a fast comparator, which feeds a PAL programmed to be a shift register that outputs RS-232 serial. The circuit is about as power hungry as it gets (shunted voltage, all analog resistors 3k). Entropy hinges on the offset voltage of the comparator and requires adjustment from counting the bit balance in the data output. I find their assumption about a fixed noise level at comparator input rather optmimistic, after I have dabbled with a noise circuit for a classic analog synthesizer; depending on the transistor batch used, these levels may vary WIDELY. (And there is the interesting story of that mysterious "selected" 2SC828 in 808s).

    The power consumption of this is absolutely not what Bunnie would have wanted. The Bunnie circuit is all about being small and low-power. There's a single stage at all, the zener noise runs straight into a modern low-voltage opamp with gain 6. I guess he wants to sample that using an on-controller A/D. The most interesting part is the TPS61158 application that ups the voltage on demand enough for an avalance zener to work. (btw: these micro-BGAs, and especially their accompanying 01005 resistors, can be mistaken for dust, and I'd rather leave those to Louis Rossman for showing off his skills.)

    I have no experience if zener noise levels are more predictable than those of abused transistors, but I wouldn't want to bet that this works reliably with different batches. If the post-A/D level too small, he can just use more samples for the needed entropy, but if the gain slams into the rails (which is something that happened to me when experimenting with a classic circuit and 1:1 values), it looks ugly.

    Starting Score:    1  point
    Moderation   +1  
       Informative=1, Total=1
    Extra 'Informative' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   3  
  • (Score: 1) by Rupert Pupnick on Tuesday April 23 2019, @01:51PM (1 child)

    by Rupert Pupnick (7277) on Tuesday April 23 2019, @01:51PM (#833846) Journal

    Thanks for that info, Rich. You raise an excellent point about keeping the noise amplitude within the input scale of the A/D. Otherwise your converter output will have lots of full scale and zero scale values. Maybe the solution to that is to simply throw those values out?

    Still the question I keep asking about the nature of the noise and randomness remains unanswered: is the power spectral density of the noise source of any importance? Why does the design you describe include high pass filter stages? What clocks the A/D circuit— does that clock have to have randomness, too? And as other posters have suggested, what is the appropriate figure of merit to use when evaluating output randomness? There’s more to good circuit design than small layout area and low power consumption.

    • (Score: 2) by RS3 on Tuesday April 23 2019, @04:42PM

      by RS3 (6367) on Tuesday April 23 2019, @04:42PM (#833905)

      I'm in a rush so I'm not thinking, but generally if you throw samples out, you radically alter the data that you want. "Digital clipping" is what we're referring to and it's a Very Bad Thing. You want to be careful with levels, and if you're not sure, use analog compression / limiters (aka AGC or Automatic Gain Control) or something similar. Simple diode limiting is no good: it will still clip and the resulting square-ish output will contain unwanted harmonics.

      In the old days of 16-bit audio sampling we had to be very very careful with input levels and clipping. Now with 24 bit and higher, we have much more "headroom", but clipping is still a Very Bad Thing (digital clipping creates a big splatter of frequencies and sounds much worse than most analog clipping). 24 bits sampling gives you 20 log (2 ^ 24) = 144 dB dynamic range (16 bits is 96 dB). In practical usage, 16 bits is very fiddly, and 24 bits is pretty easy.

      You definitely don't want random clocks! Interesting to think about though...

      In the audio world, a popular mod (hot rodding, upmod, upgrade, etc.) is to replace the sampling master clock oscillator with a much more stable (low-jitter, temperature stabilized) clock oscillator. They're not super expensive so I'm not sure why manufacturers don't do it and brag about it in the sales literature.

      The clock needs to be 2 times (plus) the highest frequency you're sampling, known as the "Nyquist rate", or you'll get aliasing.

      Best solution is good analog low-pass filters and oversample: sample rate much higher than necessary because digital bandpass filtering is easy.

      You asked about power spectral density. Yes, you want a flat long-term average (voltage squared) spectral density. Any peaks, by definition, show a particular frequency has some kind of "favoring"; better said, it's not random anymore.

      A good not too technical reference: https://www.analog.com/en/analog-dialogue/articles/practical-filter-design-precision-adcs.html [analog.com]

  • (Score: 2) by RS3 on Tuesday April 23 2019, @04:02PM (7 children)

    by RS3 (6367) on Tuesday April 23 2019, @04:02PM (#833894)

    You might be an EE, or certainly knowledgeable. Almost any small-signal circuit can be optimized for low-power consumption (and I'm sure you know that and I'm just commenting for anyone who might read this and not know).

    The first comment in this discussion contains a link to an article where the author tries some different transistors and zeners: https://betrusted.io/avalanche-noise [betrusted.io] As I commented above, I had not heard about the transistor B-E "wearout" problem, but I don't do a lot in that world- I'm usually trying to get RID of noise! I'd stick with zeners. They're meant to do that (avalanche).

    I agree about noise, batches, etc. It's not too difficult to hand-select them (or even automate it) but they may vary over long time periods anyway.

    My hunch is that the higher the zener breakdown voltage, the more noise you get.

    Since f = 1 / (2 Pi R C), the higher the zener's bias R, the more you'll get lower frequencies. And that's easily calculated. Of course you'll need an FET (very high impedance input) op amp.

    Then I would use some nice adjustable band-pass filters (parametric EQ) to get the sound I wanted. (I'm an EQ freak on soundboards, but that's my quirk / OCD).

    A solution might be to have several zeners creating the noise, each fed into 1 op amp, and the outputs summed. Noise is additive, and the more zeners you use, the flatter your spectral output should be ("should" is the favorite word of engineers, and the most hated by everyone else).

    You could use different zeners and transisitors, and then control the levels (simple mixer) and / or mute them individually.

    A good place to start might be a TL087 quad op amp.

    Or, if you don't want to mess around, you could use one of several noise generators, like the "electric druid" https://www.banzaimusic.com/electric-druid-pentanoise-noise-generator.html [banzaimusic.com]

    Or the TI SN76477 (which is out of production and superseded by the supposedly still available ICS76477).

    • (Score: 2, Interesting) by Rupert Pupnick on Tuesday April 23 2019, @09:43PM (6 children)

      by Rupert Pupnick (7277) on Tuesday April 23 2019, @09:43PM (#834051) Journal

      Yeah, I'm an EE, temporarily retired, with a few decades of hardware experience in telecom and computers. It appears that like for most of the commenters here, noise is always something you try to rid yourself of, so the objective of actually creating noise for the purpose of providing random numbers for cryptography is something I'd never seen before and immediately piqued my interest, and the first question that comes to mind is "What's the spec?". This is of course a reflection of the business culture of the commercial design environments I've worked in.

      When engineers start talking about filters, bandwidth, and signal conditioning, that generally suggests that there's some desired outcome in terms power spectral density, or the frequency content of the noise, if you like. That distribution, in turn, sets limits on the behavior of the noise signal over time, which has an obvious impact on randomness. Looking at the write up (thanks for the more direct link), it seems the emphasis is really on limiting power consumption while at the same time getting a decent output level. The scope pictures were captured on a relatively late model Tek that probably has an FFT function, and the fact that no FFT traces are presented suggests me that the designer probably isn't interested in the spectral content of the noise.

      So maybe that's not important, especially if you can post process the data that comes out of the A/D to randomize it further (e.g. do some big floating point operation, and throw out everything but the decimal remainder to get a traditional random seed number between 0 and 1).

      • (Score: 2) by RS3 on Wednesday April 24 2019, @02:01AM (5 children)

        by RS3 (6367) on Wednesday April 24 2019, @02:01AM (#834157)

        Oh, good, we can communicate. EE here too. I'm generally a Tek snob, and prefer analog, but I have a LeCroy with FFT, and an HP 4194A, as well as various audio A/D samplers- up to 192KHz @ 24 Bits and there's all kinds of software to look at spectral density. Anyway...

        Yes, the spec. I'm definitely not expert, but I remember some bits and pieces (pun intended, ugh. sorry.) So my interpretation is that the cryptographers need truly random numbers on which to base ciphers, and that any repeating patterns- the full number, or even subsections (digit combinations) that repeat more often than others, shows a potentially predictable weakness in the algorithm. Over the years I remember reading about how /dev/random is not truly random, and here and there flaws have been found in most computed pseudo-random number generators, and Holy Grail is needed but nobody has answered these questions three.

        So if we can generate true pure noise, which by definition means all frequencies represented equally when averaged over time, we can give them a good basis for encryption.

        My gut feel (as again, not expert) is that _any_ processing done won't help the randomness. I haven't done this stuff in too many years, so I have to study up on windowing, fft, etc., for either flattest end result, or, predictable result that can be mathematically compensated out. A really flat window is: Dolph–Chebyshev. Amazing write-up with nice pictures: https://en.wikipedia.org/wiki/Window_function [wikipedia.org] Let me know if you know that stuff. It's what I pursued, incl. some MS coursework but nobody would hire me- all wanted 3-5 years experience. Sigh.

        It all depends on the number of bits of "entropy" they want, but lets say 2^12. It could be based on a combination of frequencies and amplitudes, but the end result must be: long-term average histogram of all possible numbers needs to be flat-topped. Not sure what I mean by long-term, but we could derive that, and it should be based on the lowest frequency and the number of frequencies we're looking at. We could let it run for hours and see how it looks. (I do NOT need another project, but now I want to run it! Oh gosh, this is too easy not to do...)

        Again, not expert, so I'm not sure what just an amplitude histogram averaged would look like. My hunch is Gaussian- what say you?

        • (Score: 1) by Rupert Pupnick on Wednesday April 24 2019, @04:46PM (4 children)

          by Rupert Pupnick (7277) on Wednesday April 24 2019, @04:46PM (#834399) Journal

          I guess a few comments. First on my circuit design idea.

          So the noise voltage out of a 1M resistor as I suggested previously is about 0.13 uV/Hz^0.5. In a 1 MHz bandwidth (just picking an op-ampish number) that's a factor of a thousand or about 0.13 mV. To get to a 1.3V amplitude requires a gain of about 10^4 which might be doable with three gain stages with an off the shelf op amp (haven't looked at any spec sheets for GBW product for typical op amp, was guessing 50 MHz). So right away my solution's probably not cheaper. It does get you a reasonably flat spectrum, though, from the low frequency corner of the amplifier coupling to the limiting bandwidth of the system. But it's not clear to me that spectral flatness is an important or necessary condition for randomness, when I think about it some more.

          This is because as soon as you have a bandlimited signal, you establish a correlation between two successive samples in time: the lower the noise bandwidth, the more time you need between samples to make them uncorrelated. Probably not a serious functional restriction since I doubt you need to make RNG samples at MHz rates, but I think it illustrates a limitation of all noise generation systems for RNG.

          Anyway it all comes back to the question of how random is random enough for the application, which is really cryptography-- I don't have a lot of experience there. As you point out, a computationally derived RNG isn't really random because it in fact derives from a deterministic process. But practically speaking, the attacker would have to know all about the seed generation routine to defeat it (including undocumented tolerances, initial conditions and states, how long the state machine has been running, and other sources of perturbation-- e.g. data from TOY clock), and this seems really unlikely to me.

          • (Score: 2) by RS3 on Thursday April 25 2019, @06:46PM

            by RS3 (6367) on Thursday April 25 2019, @06:46PM (#834872)

            As too often, I'm in a rush and can't write up all my thoughts. But basically I wasn't going into the circuit details yet- that's just grunt work. I was trying to establish the need, philosophy, proof of concept, etc. Yes, resistors are great noise sources. "Shot noise" from any noise source could be a problem- more study needed (when I have more time...)

            1) I'm not expert but I've read here and there that good cryptography is based on truly random numbers, and any kind of repeating pattern can lead to breaking the cipher. Again, I'm not expert, but I believe them and see a need for true randomness.

            2) By definition true pure noise is composed of all frequencies within a given range of frequencies, occurring at random times and amplitudes.

            So if we can generate pure noise and correctly digitize it, we can provide true random numbers. Before getting into too many circuit details, I want to understand and define the system theory.

            One area I'm having trouble with (and might consult with a professor) is sampling. I know a fair bit about sampling, Nyquist, etc., but Nyquist isn't going to work for varying amplitudes (and I have strong thoughts about audio sampling but more another time). Issues I have: noise, by definition, contains frequencies which are varying in amplitude. That varying, also known as amplitude modulation, creates sideband frequencies which might exceed Nyquist. So to do this correctly, we need to oversample, but I'm not sure by how much, and I don't have time to think it through (gotta pay bills- wish this topic was paying my bills!)

          • (Score: 2) by RS3 on Thursday April 25 2019, @09:59PM

            by RS3 (6367) on Thursday April 25 2019, @09:59PM (#834928)

            Okay, upon further thought, I need to see that the frequencies all reach the same maximal amplitude (peak hold display), and that all frequencies have the same average amplitude, or energy, but either way, long-term average has to be a flat-topped graph. All that before I can qualify a noise source as truly random.

          • (Score: 2) by RS3 on Thursday April 25 2019, @10:18PM (1 child)

            by RS3 (6367) on Thursday April 25 2019, @10:18PM (#834936)

            Some really good reading: https://en.wikipedia.org/wiki/Hardware_random_number_generator [wikipedia.org], especially at "Quantum random properties" where they discuss quantum mechanics, molecular motion, etc., in systems above absolute zero, hence: "entropy". It's good when things connect.

            Also in that section they discuss various noise sources, frequency compensation to achieve flat spectral response, etc. If I get time I'll set up some tests and see what kind of spectral response I get from various noise sources.

            • (Score: 1) by Rupert Pupnick on Friday April 26 2019, @05:17PM

              by Rupert Pupnick (7277) on Friday April 26 2019, @05:17PM (#835198) Journal

              Good link with a nice summary of the considerations, thanks. Sounds like a 1 bit comparator into a shift register is the most elegant solution on the digital side once you have a noise source you’re happy with.