posted by
martyb
on Sunday March 27 2016, @12:27PM
from the could-this-site-run-without-both-of-them? dept.
from the could-this-site-run-without-both-of-them? dept.
Discussion on the advantages of TCP vs UDP (and vice versa) has a history which is almost as long as the eternal Linux-vs-Windows debate. As I have long been a supporter of the point of view that both UDP and TCP have their own niches (see, for example, [NoBugs15]), here are my two cents on this subject.
Note for those who already know the basics of IP and TCP: please skip to the 'Closing the Gap: Improving TCP Interactivity' section, as you still may be able to find a thing or two of interest.
It's a primer, or a refresher, or a skip. We have all kinds here. Enjoy, or don't.
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(Score: 5, Informative) by Rosco P. Coltrane on Sunday March 27 2016, @01:14PM
TCP and UDP are two different protocols, designed for two different purposes. Those who think they are in competition have skipped OSI 101.
(Score: 0, Flamebait) by patrick on Sunday March 27 2016, @02:17PM
And that's another similarity to the Linux-vs-Windows debate.
(Score: 2, Touché) by Francis on Sunday March 27 2016, @02:45PM
Not really, the only thing that Windows does better than Linux with any consistency is attract developer support.
But, whether that's worth being spied on and the generally crappy experience that's come to characterize Windows, I don't really know. That's up to the individual to decide.
(Score: 0) by Anonymous Coward on Monday March 28 2016, @10:43AM
Windows also does a better job of trampling on the user's freedoms.
(Score: 0) by Anonymous Coward on Sunday March 27 2016, @08:29PM
Less so these days, but back in the 90s a lot of the Windows vs. Linux debate centered around up-time.
Linux people were server oriented. If you were on "that other site", and you defended Windows in any way,
they automatically assumed you were defending NT Servers. Nope. I was defending the *desktop* where
4 hour up-time was "good enough" because you were probably going to go to lunch anyway. Networking and
security was an afterthought on Windows just as a responsive user-friendly GUI was an afterthought on Linux.
People just couldn't get it. It's become better over the years; but I'm sure those people are still around.
Of course these days, the debate is now totally different, with Windows now having more up-time than
I can use; but becoming a surveillance/ad platform...
(Score: 2) by bitstream on Monday March 28 2016, @08:25AM
Having a reboot etc when you have 10s of terminals and stuff up right in debugging is royal pain. So operating systems that can't stay stable will not be acceptable. It's also a stress to keep saving all the time because.. maybe.. ¤%¤%.
(Score: 2) by RamiK on Sunday March 27 2016, @05:13PM
Well, yes and no. Games and web apps have mixed needs. Some things need low latency (coordinating GUI elements) while other things can wait in line (loading files). Often, developers will customize their own UDP protocol under these circumstances to handle everything efficiently. But, even the best designed protocol will still crush and burn in the face of the ISP's infrastructure down-prioritizing anything not VOIP. Your awesome custom protocol might even lose out to BitTorrent or Windows 10 P2P Update protocol since you're not one of the cool kids that get Cisco to override your header by default... So, the "stupid" solution of using UDP for the GUI while opening a second TCP connection for loading files will often be the smartest and best thing to do, even if it means more client side complexity coordinating it all.
So, yeah... They're designed for two different purposes. But things get messy in real life.
compiling...
(Score: 2) by bitstream on Monday March 28 2016, @08:29AM
Fake your packets to come from one of the "cool kids" ..?
Robotics control over VoIP.. why not?
Dunno what your specific application you have in mind.
(Score: 2, Insightful) by Anonymous Coward on Sunday March 27 2016, @06:20PM
I want to be able to control the things TCP mandates I must have, but that I don't neccesarily need. I need a reliable channel that can be "reset" cheaply according to application events: The fullscreen application has changed, no need to ensure packets containing the old data still in transit reach the endpoint -- and I want the new data stream to replace the old stream in the TCP packet window.
This is not a UDP need. This is me needing TCP to be more flexible. However, the designers gave us the inflexible TCP and said: If your use case is not exactly the way we built TCP to handle, fuck you. There are no application level configuration options, eat shit and go re-invent the fucking wheel using UDP, or otherwise suffer with kludgey workarounds like opening multiple TCP connections between the endpoints (placing an exponential resource load on the server for no fucking reason other than lack of config options).
It's like the devs who hardcode the game controls to WASD, and say: Fuck all you people without QWERTY Keyboards; US Latin 1, or death! It's assinine and amazingly simple to have a config that says: Press the key you want for forward. Press the key you want for left, etc. TCP should have done that. It didn't. Therefore I have implemented (a slightly modified) TCP on top of UDP, and so has Google and a shit load of other people.
It's like saying: It works for me! There is a TCP / UDP debate. The debate is whether TCP should be changed to be a bit more like UDP to improve everyone's experience by overcoming the short sightedness of the original design. And people like Google and myself are actually furthering the debate. Idiots like yourself who say "there is no debate" are the enemies of progress.
Jump in a tar pit gramps, you're hindering the herd.
(Score: 3, Informative) by maxwell demon on Sunday March 27 2016, @06:50PM
FTFA:
There's no way to tell the routers in between to drop the already-sent packets because the routers forget each packet as soon as it is sent to the next router. You don't even have the guarantee that all your packets go the same way; how would a router on route 1 know that a cancellation message was delivered using route 2?
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by Geotti on Monday March 28 2016, @01:33AM
Don't you get it? "you're hindering the herd"
These "progressive" kids are just chasing the wrong kind of efficiency. Let them make their own internet and let's continue to use ours, unless they start to impose their shit on us like that Poettering dude.
(Score: 2) by isostatic on Monday March 28 2016, @10:27AM
TCP and UDP are two different protocols, designed for two different purposes. Those who think they are in competition have skipped OSI 101.
So I'm building a new application which requires a new communication protocol. I can use something based on a high level protocol like SOAP or a message broker, I could use something based on something lowerdown like http, or I could base it on UDP or TCP. Going down to raw IP datagrams becomes problematic. TCP comes with advantages, but also drawbacks, but the same can be said about SOAP, or Corba, or http, or snmp.
They aren't as different as a hammer and screwdriver, more like a screw vs a nail, they're both designed for the same purpose (fixing things together // communicating over a network), but they do it in different ways with different advantages and disadvantages.
If I wanted to send messages to many clients, I could use a message broker running on tcp to ensure the message gets there. On the other hand if I'm sending a broadcast message saying "new information, update now rather than wait 10 seconds for the poll), why wouldn't I just fire out a broadcast udp?
Packet loss is a bit of a red herring. At any one time I'm streaming 250mbit around the world on the internet, and I see packet loss post-RTP-reconstruction and minimal buffering in the 1-in-1,000 - 1-in-10,000 level.
(Score: 2) by driverless on Wednesday March 30 2016, @06:26AM
We have all kinds here.
Oh, we got both kinds, we got TCP and UDP.
(Score: 3, Interesting) by pvanhoof on Sunday March 27 2016, @01:26PM
Improving interactivity in TCP/IP protocols is usually done with pipelining to make the protocol less chatty. I scanned the guy's two cents, but his article apparently doesn't even mention this?
I agree that most client developers still ASK, WAIT, RECEIVE, ASK, WAIT, RECEIVE while they can do 01 ASK\n02 ASK\n 03 ASK to get 01 ANSWER\n02 ANSWER\n 03 ANSWER. Feel free to sniff most E-mail clients's traffic pattern.
Is that maybe because HTTP, where everything is tunneled over nowadays, sucks at pipelining? Have people forgotten?
(Score: 5, Funny) by McD on Sunday March 27 2016, @02:08PM
You know what the great thing is about UDP jokes?
.
.
.
I don't care if you don't get them!
#include <rimshot.h>
I'm here all week, tip your waiters!
(Score: 5, Funny) by patrick on Sunday March 27 2016, @02:26PM
"Can I tell you a TCP Joke?"
"Yes, please tell me a TCP Joke."
"Ok, I'll tell you a TCP Joke."
--------------------------------------------------
If you didn't understand the TCP joke, I'll keep telling it slower and slower until you get it.
(Score: 5, Funny) by ikanreed on Sunday March 27 2016, @03:52PM
"Okay, I'll tell you a UDP Joke"
"Can I tell you a UDP Joke?"
(Score: 0) by Anonymous Coward on Sunday March 27 2016, @08:08PM
"Can I tell you a UDP Joke?"
"I thought we were talking about cats and T. Gondii?"
(Score: 2, Funny) by Francis on Sunday March 27 2016, @02:48PM
You know what the great thing about UDP jokes is?
.
.
.
.
.
.
.
.
(Score: 0) by Anonymous Coward on Sunday March 27 2016, @02:12PM
Hey guys, please stop disabling my algorithm, OK? I'm here to help!
(Score: 2) by isostatic on Sunday March 27 2016, @02:57PM
But I can sell suits snakeoil promising to transfer your 1000mbit file over your 100mbit connection in 9 seconds, where under fcp it takes 32 years
(You find their demos have disabled tcp window scaling and they transfer easily compressible video files like dvc with silent audio in an mxf wrapper and compare with a protocol that doesn't have built in compression)
But it's enough that idiotic "architects" buy the product, refuse to run in house tests because "they've already bought it", and then insist you use it rather than a nice simple sftp despite it being slower.
(Score: 0) by Anonymous Coward on Sunday March 27 2016, @03:15PM
While I agree overall, the problem comes when it's time to send lots of small amounts of data very fast. Market data applications generally want updates as fast as they can, not as efficiently as they can. When dealing with differential updates, you only maybe have a couple bytes per symbol that change per update, and might not have many of those at a time, but the expectation is that you better be able to receive them before they get sent sometimes.
Admittedly a niche situation, but kind of a deal breaker when comparing two software products that do the same thing and one consistently outperforms the other by ~1-5 ms. I've seen that one happen first hand both in a clean environment and at client sites. ITG comes to mind WRT some of their Reuters systems that fed (what I suspect was) their algo trading systems. They wouldn't ever let me get even a hint of what they were consuming data with, but it was home-grown and very protected. Kind of a bitch when trying to diagnose issues.
Posting AC because, well, the name drop, and I don't really like being associated with that kind of work. I guess it's not killing people for a living, but it's really not a whole lot better. I don't do that anymore.
(Score: 0) by Anonymous Coward on Sunday March 27 2016, @06:38PM
* Even not that long ago it seemed more of a stupid hack than a real solution. When your "solution" shows up as a top cause for many TCP performance problems, you should admit you've screwed up.
(Score: 0) by Anonymous Coward on Sunday March 27 2016, @05:01PM
udp attributes: shout. general idea/topic (of the conversation) is known. like: shine a light into a dark room -aka- the network.
tcp attributes: argument. state-teleportation. requires duplication for operation. like:create a "light-emitting display" to show the time.
thoughts:
to test transmit media connection (copper, fiber, radio-waves, etc.) should use udp (how reliable is the connection? we don't know until we test. too much testing might fail the connection. be gentle); to fall-back on udp-test-confirmed reliable connection to TCP?
both tcp and udp are "for the network". it is assumed that computers are first and foremost ALONE/singular/not-networked.
the network-enabled program can do alot inside the payload part of tcp or udp.
a "lazy" networked-programs relies alot on tcp/udp;
a "smart" networked program doesn't get the "compression"
benefit of a "lazy" program and might have to implement/duplicate
some tcp/udp functions in the payload part ...
(Score: 2) by maxwell demon on Sunday March 27 2016, @06:59PM
Isn't that what ICMP is for?
The Tao of math: The numbers you can count are not the real numbers.
(Score: 0) by Anonymous Coward on Monday March 28 2016, @02:56AM
Yes but people routinely drop ICMP packets on the floor. With the DNS amplificaton attacks, UDP is becoming the next ICMP [soylentnews.org].
(Score: 3, Touché) by Craig Cherry on Sunday March 27 2016, @05:17PM
"a history which is almost as long as the eternal Linux-vs-Windows debate"
It's worth knowing that these network standards date primarily from the minicomputer era, when the very simple PCs of the time tended to use slow point to point serial port connections.
UDP was defined in RFC768 in 1980. TCP dates from a few years earlier, so presumably the first discussions about when to use TCP versus UDP would have been in 1980.
Microsoft announced Windows 1.0 in 1983 and shipped it in 1985.
Linux started in 1991.
(Score: 0) by Anonymous Coward on Monday March 28 2016, @09:39AM
Whatever. Since all those things happened long before any of us were born, their exact chronological order only matters to pedantic historians and trivia buffs.
(Score: 2) by devlux on Sunday March 27 2016, @05:28PM
There is an old adage which goes "Those who do not learn TCP are doomed to reimplement it, poorly".
Apologies to Henry Spencer https://en.wikipedia.org/wiki/Henry_Spencer [wikipedia.org]
(Score: 2) by Common Joe on Sunday March 27 2016, @05:50PM
I just went to a talk this week where one of the lecturers was explaining how their database system relied on UDP instead of TCP. I looked at him as if he were from Mars. Realizing I wasn't fully following, he explained something I didn't know and thought I'd share here.
His company works on a distributed system based on a heavily modified distributed PostgreSQL network; they call it a "massive parallel, shared-nothing database". There are one or two machines that sit out front and a break the query down so that other PostgreSQL systems can parse those newly formed statements. The queries execute in parallel (but share no resources in any way), the front end machines get the information back, and they reassemble the results and send it on to the originator of the queries. They claim the databases can be hundreds of TBs or even PBs worth of information.
They use UDP to do it, though. I was shocked because I thought packet reliability was going to be king with this kind of database setup. He explained that when scaling to these sizes, the resources for hundreds or thousands of open sockets for TCP/IP communication was too expensive for those machines up front. The UDP fire and forget method was much better for resources. They do, of course, have checks on those UDP packets and if they don't get results back in a predetermined amount of time, they will resend, switch to failover systems, or do other appropriate things. Apparently, packet loss isn't a big problem since the UDP packets are sent on a closed network. He says very few packets are ever lost.
I found that thought very interesting. I never imagined using UDP that way, but then I also understand the reasons why they did it and it's not a typical database setup.
He put his slides online, but I purposely didn't post it here. I wasn't looking to post a Soyvertisement (I don't know anything about his company anyway), it looks like it's his personal website, and the talk he gave was not about TCP/IP vs UDP so there is nothing about it in his slides. The slides are mainly about how they took their program and made it open source... but he had to explain his product first which is how we got onto the TCP/IP vs UDP bit. Out of the 91 slides, only a few are dedicated to the setup of the parallel processing database setup. If you're just dying to see the slides, google "massive parallel, shared-nothing database" with quotes and you can't miss it. You can also email me and I'll send you the link. Due to craziness in my life, it may take me a couple of days to respond, but I'll try to do a decent turn around.
(Score: 2) by devlux on Sunday March 27 2016, @06:14PM
I dunno about his particular implementation. However UDP is fine as long as you don't need guarantees. Once you start needing guarantees then you begin to start re-implementing TCP. If you own the entire loop, i.e. it's a private network this might be worthwhile but as soon as you begin to expand beyond the local side of things TCP does become king. Turning Nagel off on TCP can help alot if you're mostly smallish updates and need closer to realtime performance, but most of the features of TCP are there for a reason. That reason is to make sure your data arrives at it's destination. If your data only needs to "mostly" arrive then UDP becomes superior.
What I don't understand is why no one has ever implemented a packet multicasting option for UDP as a standard.
This would make UDP a lot more useful since a lot of people could subscribe to a a feed and it wouldn't be limited to the the upstream of the originator.
(For those who don't know, the reason your video conference craps out at 4 or 5 people has little to do with your downstream bandwidth. if you have 5 people in a videochat your computer is pumping out the exact same packet 5 different times, simply changing the destination IP. What I'm saying is this seems like a function the router would be better at, failing that, having multiple ip destinations in a single packet might be good although I can see the IP header space quickly swamping the data portion in a large enough conferencing app.)
(Score: 2) by c0lo on Sunday March 27 2016, @08:52PM
Ah, such a simple solution, how come others haven't thought of it [wikipedia.org]?
(grin)
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 2) by devlux on Monday March 28 2016, @03:15PM
Thanks I guess what I meant is as part of a major working standard like WebRTC.
(Score: 0) by Anonymous Coward on Monday March 28 2016, @10:57PM
(Score: 2) by Common Joe on Monday March 28 2016, @07:14AM
My understanding is that that is the key. The network is private so they don't lose tons of packets -- just a few. It sounds like their program can handle those few losses. And you're right, if they opened up the network beyond that, the setup sounds like it would fail. Obviously, this is not a standard database setup, but it's an interesting enough idea that it could be used for other extremely specialized programs. And for all I know, other programs may use it.
(Score: 2) by darkfeline on Sunday March 27 2016, @06:36PM
Or you can post it here because you thought it was interesting.
Please do not emulate the horrible forum practice of:
"I need help!"
"I PMed you the fix =)"
Join the SDF Public Access UNIX System today!
(Score: 2) by Common Joe on Sunday March 27 2016, @08:01PM
I thought his talk and the comments about TCP/IP and UDP in the talk were interesting. That's all. On the slides are a few diagrams of their setups. I don't need any help with anything as I'm not affiliated with them and have no project I'm currently working on like this. If I want something from those on Soylent News, I won't be bashful or beat around the bush about asking. I quite believe these people are smarter and more experienced than me concerning networking. As far as the details as to what they did, he didn't get into it much more than what I wrote. I can see what they did, though. I'll try to write some further details and thoughts to devlux's reply tomorrow after I've had a night's sleep.
(Score: 0) by Anonymous Coward on Sunday March 27 2016, @08:56PM
You can build reliability on top of UDP, but it's a lot of work to get it right. Obviously, that's what they signed up to do, unless they're using a third party protocol library.
There's the risk of starting on the path of re-inventing TCP to deal with all the pitfalls the original TCP developers ran into over 20-25 years. For example, flow control; what if the master node (the one dispatching the requests) runs short of message buffer space? TCP has a solution for that, UDP doesn't.
(Score: 4, Informative) by Geotti on Monday March 28 2016, @02:13AM
I'll make it easier for you: it's greenplum [greenplum.org], here [scherbaum.la] are the slides, and here's their github [github.com]. ;)