SoylentNews is people

Title    Sometimes the Noise is Signals, Too
Date    Tuesday December 13 2016, @07:46AM
Author    Fnord666
from the come-on-feel-the-noize dept.

charon writes:

This insight into the information which can be gleaned from data is cool and worrisome by equal measures.

Early in his talk, computer scientist John Hopcroft noted a funny fact about clustering algorithms: they work better on synthetic data than real data. But this is more than an odd tidbit about software.

[...] When we invent our own synthetic data, we try to mimic real data by mixing true information with random distraction–combining "signal" with "noise." But in real data, the divide isn't so clear. What often looks like noise turns out to be the deep structure we haven't grasped yet.

Hopcroft's insight: data doesn't just have one structure. It has many. If I scanned notebooks from a hundred people, and made a database of all the individual letters, I could sort them lots of ways. Alphabetically. Capital/lowercase. Size. Darkness. Handwriting. Each of these is a different layer of structure.

And to understand data–and the world–you've got to reckon with all those layers.

The part of the video which discusses the above starts around 5:45.

Original Submission


  1. "charon" -
  2. "cool and worrisome" -
  3. "video" -
  4. "Original Submission" -

© Copyright 2021 - SoylentNews, All Rights Reserved

printed from SoylentNews, Sometimes the Noise is Signals, Too on 2021-09-18 23:14:00