OpenBSD developer, Gilles Chehade, debunks multiple myths regarding deployment of e-mail services. While it is some work to deploy and operate a mail service, it is not as hard as the large corporations would like people to believe. Gilles derives his knowledge from having built and worked with both proprietary and free and open source mail systems. He covers why it is feasible to consider running one.
I work on an opensource SMTP server. I build both opensource and proprietary solutions related to mail. I will likely open a commercial mail service next year.
In this article, I will voluntarily use the term mail because it is vague enough to encompass protocols and software. This is not a very technical article and I don't want to dive into protocols, I want people who have never worked with mail to understand all of it.
I will also not explain how I achieve the tasks I describe as easy. I want this article to be about the "mail is hard" myth, disregarding what technical solution you use to implement it. I want people who read this to go read about Postfix, Notqmail, Exim and OpenSMTPD, and not go directly to OpenSMTPD because I provided examples.
I will write a follow-up article, this time focusing on how I do things with OpenSMTPD. If people write similar articles for other solutions, please forward them to me and I'll link some of them. it will be updated as time passes by to reflect changes in the ecosystem, come back and check again over time.
Finally, the name Big Mailer Corps represents the major e-mail providers. I'm not targeting a specific one, you can basically replace Big Mailer Corps anywhere in this text with the name of any provider that holds several hundred of millions of recipient addresses. Keep in mind that some Big Mailer Corps allow hosting under your own domain name, so when I mention the e-mail address space, if you own a domain but it is hosted by a Big Mailer Corp, your domain and all e-mail addresses below your domain are part of their address space.
Earlier on SN:
Protocols, Not Platforms: A Technological Approach to Free Speech (2019)
Re-decentralizing the World-Wide Web (2019)
Usenet, Authentication, and Engineering - We Can Learn from the Past (2018)
A Decentralized Web Would Give Power Back to the People Online (2016)
Decentralized Sharing (2014)
Related Stories
Anonymous Coward writes:
""MediaGoblin is a free software media publishing platform that anyone can install and run. Decentralization, (...) is the main goal of the project, one that is backed and connected to the GNU project.
So far, MediaGoblin has raised only $3,000 of its $60,000 goal, with the campaign set to end April 14th, (...) that is a date that is soon approaching. The first crowd-sourcing initiative was in October of 2012, so this is not the first crowd-funding initiative the project has launched. This second campaign was clearly spurred on by the PRISM revelations of recent past. Having not noticed any failures to meet 2012's funding campaign, it's very possible the team may reach their goal again, given the intensity of the subject matter."
The original purpose of the web and internet, if you recall, was to build a common neutral network which everyone can participate in equally for the betterment of humanity. Fortunately, there is an emerging movement to bring the web back to this vision and it even involves some of the key figures from the birth of the web. It's called the Decentralised Web or Web 3.0, and it describes an emerging trend to build services on the internet which do not depend on any single "central" organisation to function.
So what happened to the initial dream of the web? Much of the altruism faded during the first dot-com bubble, as people realised that an easy way to create value on top of this neutral fabric was to build centralised services which gather, trap and monetise information.
[...] There are three fundamental areas that the Decentralised Web necessarily champions: privacy, data portability and security.
Privacy: Decentralisation forces an increased focus on data privacy. Data is distributed across the network and end-to-end encryption technologies are critical for ensuring that only authorized users can read and write. Access to the data itself is entirely controlled algorithmically by the network as opposed to more centralized networks where typically the owner of that network has full access to data, facilitating customer profiling and ad targeting.
Data Portability: In a decentralized environment, users own their data and choose with whom they share this data. Moreover they retain control of it when they leave a given service provider (assuming the service even has the concept of service providers). This is important. If I want to move from General Motors to BMW today, why should I not be able to take my driving records with me? The same applies to chat platform history or health records.
Security: Finally, we live in a world of increased security threats. In a centralized environment, the bigger the silo, the bigger the honeypot is to attract bad actors. Decentralized environments are safer by their general nature against being hacked, infiltrated, acquired, bankrupted or otherwise compromised as they have been built to exist under public scrutiny from the outset.
In the Web 3.0 I want a markup tag that delivers a nasty shock to cyber-spies...
Professor Steve Bellovin at the computer science department at Columbia University in New York City writes in his blog about early design decisions for Usenet. In particular he addresses authentication and the factors taken into consideration given the technology available at the time. After considering the infeasiblity of many options at the time, they ultimately threw up their hands.
That left us with no good choices. The infrastructure for a cryptographic solution was lacking. The uux command rendered illusory any attempts at security via the Usenet programs themselves. We chose to do nothing. That is, we did not implement fake security that would give people the illusion of protection but not the reality.
For those unfamiliar with it, Usenet is a text-based, worldwide, decentralized, distributed discussion system. Basically it can be likened to a bulletin board system of sorts. Servers operate peer to peer while users connect to their preferred server using a regular client-server model. It was a key source of work-related discussion, as well as entertainment and regular news. Being uncensorable, it was a key source of news during several major political crises around the world during the 1980s and early 1990s. Being uncensorable, it has gained the ire of both large businesses and powerful politicians. It used to be an integral part of any ISP's offerings even 15 years ago. Lack of authentication has been both a strength and a weakness. Professor Bellovin sheds some light on how it came to be like that.
Despite weaknesses, Usenet gave rise to among many other things the now defunct Clarinet news, which is regarded to be the first exclusively online business.
Researcher Ruben Verborgh explains how to re-decentralize the World-Wide Web, for good this time. He argues that decentralization is foremost about choice and thus people should be free to join large or small communities and talks up Solid as a primary option.
Originally designed as a decentralized network, the Web has undergone a significant centralization in recent years. In order to regain freedom and control over the digital aspects of our lives, we should understand how we arrived at this point and how we can get back on track. This chapter explains the history of decentralization in a Web context, and details Tim Berners-Lee’s role in the continued battle for a free and open Web. The challenges and solutions are not purely technical in nature, but rather fit into a larger socio-economic puzzle, to which all of us are invited to contribute. Let us take back the Web for good, and leverage its full potential as envisioned by its creator.
Earlier on SN:
Tim Berners-Lee Launches Inrupt, Aims to Create a Decentralized Web (2018)
Decentralized Sharing (2014)
Mike Masnick, usually editor for Techdirt, has written an essay on a technological approach to preserving free speech online in spite of the direction things have been heading in regards to locked-in platforms. He proposes moving back to an Internet where protocols dominate.
This article proposes an entirely different approach—one that might seem counterintuitive but might actually provide for a workable plan that enables more free speech, while minimizing the impact of trolling, hateful speech, and large-scale disinformation efforts. As a bonus, it also might help the users of these platforms regain control of their privacy. And to top it all off, it could even provide an entirely new revenue stream for these platforms.
That approach: build protocols, not platforms.
To be clear, this is an approach that would bring us back to the way the internet used to be. The early internet involved many different protocols—instructions and standards that anyone could then use to build a compatible interface. Email used SMTP (Simple Mail Transfer Protocol). Chat was done over IRC (Internet Relay Chat). Usenet served as a distributed discussion system using NNTP (Network News Transfer Protocol). The World Wide Web itself was its own protocol: HyperText Transfer Protocol, or HTTP.
In the past few decades, however, rather than building new protocols, the internet has grown up around controlled platforms that are privately owned. These can function in ways that appear similar to the earlier protocols, but they are controlled by a single entity. This has happened for a variety of reasons. Obviously, a single entity controlling a platform can then profit off of it. In addition, having a single entity can often mean that new features, upgrades, bug fixes, and the like can be rolled out much more quickly, in ways that would increase the user base.
Earlier on SN:
Re-decentralizing the World-Wide Web (2019)
Decentralized Sharing (2014)
(Score: 5, Interesting) by Arik on Monday September 02 2019, @09:51AM (9 children)
Never officially confirmed. Feel free to chime in with confirmations and/or disproofs.
I think they laid off everyone that understood how the internet worked about 10 years ago. As things have broken they have outsourced them.
To people that don't understand how the internet worked. Or how to make anything other than their paycheck work.
If laughter is the best medicine, who are the best doctors?
(Score: 5, Interesting) by NateMich on Monday September 02 2019, @12:07PM (6 children)
I'd like to disagree with you, since I work at a fairly large ISP.
But now that I think about it, so many of our good people have left in the last few years and we have indeed been replacing them with outsourced support. Also we kind of suck now.
Yeah, I think you might have a point.
(Score: 2) by RS3 on Monday September 02 2019, @04:10PM (4 children)
I've used Verizon personally as an ISP for 20 years, and professionally for 12.
Looking back, I was so wrong when I thought they sucked 15 years ago. I had _no_ idea how much suckier they would aspire to. I've never hated AOL- it has served its market well, but not for me. When Verizon bought them and Yahoo!, then moved verizon.net email addresses to AOL, which are actually run on Yahoo! servers, things became annoying. Basically something they did disabled my pop3/smtp client, but they _refused_ to admit they did something to change things.
Professionally, I was adminning some email servers. One I inherited was based in qmail, which needed to die anyway. It did also host Squirrel Mail (webmail) which I recall being pretty awesome (as much as webmail can be).
I built ones using sendmail and postfix (don't read into that- different machines- some send only, and all are on postfix, but it's now a moot point...).
Verizon "customer support" is horrific. What they first did is block all port 25 smtp traffic, instead opening port 587, which required authentication to then relay packets. No problem with sendmail / postfix, but qmail had port 25 hard-coded everywhere. I actually tried to find and replace every instance of port 25, but I could never get it to compile- not even from clean raw unchanged source. That was while I was under great pressure from customers who based their businesses on a now broken email. Within 1 day my boss moved everyone to a godaddy account. He's really smart that way; although I think godaddy is crap, at least he kept his customers.
So then Verizon added some more layers of encryption and authentication, and sendmail / postfix handled it.
Until, they now scan the "from" field in email headers, and flat-out refuse to relay any email packet that does not have a known valid verizon.net email address.
Let's say you have a webserver hosting small local business's sites, and on those sites are some webforms which call up a script to send an email for, oh, maybe an order for a food catering business, or whatever. You want the email to look like it came from the prospective customer, so when the business owner / representative gets the inquiry email, they just hit "reply" and the "to" field is filled in correctly. Nope, Verizon won't allow that scenario. Comcast will, and maybe someday the boss will let me move to Comcast.
Postfix is pretty cool.
(Score: 2) by hwertz on Monday September 02 2019, @05:53PM (3 children)
"You want the email to look like it came from the prospective customer, so when the business owner / representative gets the inquiry email, they just hit "reply" and the "to" field is filled in correctly. Nope, Verizon won't allow that scenario. Comcast will, and maybe someday the boss will let me move to Comcast."
It's an oversite on Comcast's part to NOT block this. Think about it this way... "A greasy spammer wants the e-mail to appear to come from some random schlub, so when the spam receiver who doesn't look at e-mail headers complains they complain about the wrong address. Yep, shockingly Comcast allows this."
The setup you had is convenient, but there's very good reasons for it to not be allowed, not just to inconvenience you.
(Score: 3, Interesting) by RS3 on Monday September 02 2019, @06:43PM (2 children)
Yup, I'm very smart, I know this. But how do you fix it? I tried using the "reply to" field but most email clients don't honor it. Everyone has to have a verizon.net account?
Or we just break email and maybe the whole internet just because there are bad actors out there? Nevermind that the technology exists to trace their IP- they have to be connected through an ISP somewhere in the world. So punish everyone for the wrongdoings of a few? I call that laziness on the part of the "authorities" and advocate replacing them.
In my case, the "greasy spammer" has to manually fill in the form, and there might have been re-captchas on them too, so there was very little spam.
Oh, and Verizon have implemented very effective spam filtering long ago. It's computers (servers) running scripts and spam scanners. So I don't understand what all the whining is about.
(Score: 1, Informative) by Anonymous Coward on Tuesday September 03 2019, @02:32PM (1 child)
You need to use Mailgun or some other special service. You will also need to configure whatever authority dujour is popular and required in DNS to sanction it or all your mail will end up blackholed. I think DKIM or DMARC is the new hawtness for that.
https://en.m.wikipedia.org/wiki/DomainKeys_Identified_Mail [wikipedia.org]
The requirements seem to change every few months.
(Score: 2) by RS3 on Wednesday September 04 2019, @01:11AM
Yes, and thanks for the info. In fact I looked into many SMTP relay providers, including Mailgun, but the company owner would not pay for them. It's a low-budget tight business, and probably his smallest / least significant. Years ago he hosted some very very major websites, but businesses change like the wind, he lost the accounts, and moved his time and attention to other things (that are very successful). Of course I'd love to grow the hosting business, but I'm not a marketing / sales-type. And it's impossible to compete with the godaddys of the world.
(Score: 3, Interesting) by PartTimeZombie on Monday September 02 2019, @10:07PM
The ISP I deal with for work has about 50% of the market in my country.
They offered all their staff redundancy earlier this year. Guess who took them up on the offer?
Yes, all the staff who knew anything. I dread having to get ask of their "support" people to do any troubleshooting.
(Score: 2, Touché) by Anonymous Coward on Monday September 02 2019, @02:59PM
It makes a degree of sense. When things are (relatively) new and untested you need a greater percentage of people who understand how everything works. Get to a level of developmental stability, and unless you are introducing something truly novel you no longer need the expense of the people who understand everything. I'm assuming that those who understand everything would be getting greater compensation for their greater degree of knowledge. Once the factory has been built you need people who understand which buttons to push, not necessarily how to make all the machines - you call in maintenance people when you need to because on-staff maintenance are not cost effective.
It might be that they retained some people who understood everything but those people are now overloaded either supervising or otherwise filtering the work of those who have more incomplete knowledge.
This can then devolve into people understanding less and less, and a race to the bottom of personnel cost and concurrent knowledge.
(Score: 2) by legont on Monday September 02 2019, @08:22PM
Can't say about ISP's, but in finance it is definitely true. The only objection of mine, while they did outsource, that part of humanity knows even less.
There will be blood; on the street that is.
"Wealth is the relentless enemy of understanding" - John Kenneth Galbraith.
(Score: 5, Funny) by coolgopher on Monday September 02 2019, @10:18AM
You know, I'd take the nettles over sendmail.cf any day.
(Score: 3, Insightful) by Anonymous Coward on Monday September 02 2019, @11:23AM (13 children)
It's dealing with all the stuff in place to prevent spam (you have to get a bunch of stuff signed, and Microsoft's basically doesn't work at all but you still need it to talk to anything using their email server), then you have to get your isp to allow access to port 25 - forget doing it at home, and even most hosting companies are touchy about it - and then you have to provide clients that people want to use, because there's no free equivalent to Gmail.
(Score: 4, Insightful) by NateMich on Monday September 02 2019, @12:14PM (1 child)
The users are the hard part.
(Score: 2) by PartTimeZombie on Monday September 02 2019, @10:22PM
I modded you +1 Touche, but could just have easily been Insightful, Informative, or Funny.
(Score: 5, Interesting) by Grishnakh on Monday September 02 2019, @02:38PM (8 children)
This is it right here.
Even if you didn't have to worry about the roadblocks and could just set up your own SMTP server, actually using it for mail is a waste of time unless you redirect it all to your GMail account, because otherwise you'll just be spending all your time sifting through spam. Spam has almost completely ruined email.
(Score: 0) by Anonymous Coward on Monday September 02 2019, @05:02PM (2 children)
It has completely ruined traditional email. There would still be hope with user definable addresses that can be dropped as soon as the spammers pick up on them, perhaps combined with a postage system where a token must be included in the message.
Even if the postage were free, it could be useful to have human involvement before a sender is allowed to pass email: Answer these random math questions and include the string it sends back to you in your email.
Both the user definable addresses and the postage system can permit mail to be rejected during the transaction.
(Score: 1, Informative) by Anonymous Coward on Tuesday September 03 2019, @02:05AM
Your post advocates a
(X) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
(X) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(X) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(X) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
(X) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
(X) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
(X) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
(X) Outlook
and the following philosophical objections may also apply:
(X) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
(X) Countermeasures must work if phased in gradually
(X) Sending email should be free
(X) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!
(Score: 1, Informative) by Anonymous Coward on Tuesday September 03 2019, @02:37PM
Postage for email system using math has existed for a long time. It was called HashCash basically a bitcoin like proof of work system for email that predates bitcoin by several years.
(Score: 2, Touché) by Anonymous Coward on Monday September 02 2019, @11:04PM (3 children)
Nope. Run my own email, on my own server, on my home internet link. Have been doing so since circa. 2000 (so about 19-20 years or so now). My main email address has been unchanged in all that time.
I get, maybe, 1 spam getting paste the filter every 6 months. Otherwise, Crm114 cleans all the rest up and I simply don't see any spam.
And that one every 6 months, feed it back into crm114 so it can learn, and wait another six months.
So, no, one will not spend all their time sifting through spam, provided one knows to use a good spam filter.
(Score: 1) by nekomata on Tuesday September 03 2019, @05:45PM (1 child)
I have been running OpenBSD with OpenSMTP for 5-ish years. My SPAM filtering is just greylisting and bgpd based black/whitelisting (http://bgp-spamd.net/). I get literally zero spam mails. Not kidding, I can't remember getting a single spam email since I set this up. Also the setup is generally pretty good, I don't have problems with getting into other ppls spam boxes etc.
The whole setup takes a weekend, and then an openbsd update every 6 months wich is the most painless, best documented system upgrade I have ever experienced. YMMV of course, but I have not found a personal email server to be a hassle at all.
(Score: 0) by Anonymous Coward on Wednesday September 04 2019, @04:25AM
One of my coworkers was half a beat from pulling the trigger on changing our our mail server for a hosted solution, despite the fact that we run plenty of other servers. He happened upon an article about greylisting. He set it up, and it made a huge difference according to him. Apparently, just the act of delaying mail keeps most spammers from trying again. In addition, it also buys just enough time for other automated anti-spam systems to flag the sender as suspicious. In addition, most addresses where we need an email RIGHT NOW, send us email often enough that they don't get caught by the greylist.
(Score: 1) by DECbot on Tuesday September 03 2019, @06:57PM
I have Comcast run my spam filter for the mail server in my basement. I get zero spam, just updates from my FreeBSD server whining about 'update this' and 'new release that' and 'the raspberry pi detects that you're out of salt in your water softener tank'. Though I've noticed there are a lot of false positives with Comcast's filtering. Perhaps I should get around to asking them to stop blocking port 25 or set up my server and router for port 587, but then I'd have to do something about the spam.
cats~$ sudo chown -R us /home/base
(Score: 3, Interesting) by Hyperturtle on Tuesday September 03 2019, @02:45PM
I am not sure how to best reply, but I am disappointed when I read that people are willing to let an advertiser block ads because it is too hard to set up something themselves.
It isn't hard nor inconvenient; but there are obstacles that can stymie one's chances for success.
Email addresses themselves don't just start receiving advertisements -- they have to be harvested or handed out. If you create an "amazon@mydomain.wtf" account on your email server hosting mydomain.wtf, and only use it for amazon, you will get only amazon and their marketplace seller info. If you get anything else, chances are someone there sold you out. I have numerous accounts and no I don't have them all linked on my phone. I check then when I need to. I have accounts that I've sent to and from the internet and other servers for years and years, and never have once received an unsolicited message. It can be done, but if that is too inconvenient, then it can't be done for people unwilling to take the steps necessary to keep things private.
If you have an email address for just here -- for just ebay, for just facebook, for just this or that, you can significantly diminish the amount of spam, and identify where it comes from. If you use one adress for everything, or just a few for everything else, they're going to get spammed and keep getting spammed because you're regualrly using them for everything and/or everything else.
And for the love of god, do not use html in email unless you have some sort of non-email related network edge filtering to block well-known 1x1 single pixel.gif server hosts. Once your carefully protected email address is used in webmail or an html rendering email client to pull down that "LOOK A LIVE EMAIL ADDRESS CHARGE!!!" pixel, there is no undo button. That is hard to track over the long term, so the best bet is to not enable HTML in your email client unless you have a good reason to for a specific message.
These controls are not easy for someone like my mom to abide to, but I still don't steer her to google... even if it is easy to become a host for the internet's version of the cordyceps mushroom of email ecosystem content harvesting.
That all stated, and I didn't really explain much...I can't get into ISP stuff or actual effort involved in setting up a server--the article covers much of that ground anyway. I fully agree that there are inconveniences, ISPs that get in the way, the matter of hosting such a server, setting it up. It's all a value proposition. What is it worth to a person to be in control? To accept that maintenance may need to be done from time to time? I am not the type to throw in the towel and let an advertising company manage the filtering of unwanted ads, but that is just me... I'd much rather get ads by mistake than by design. And it really hurts when a preventable situation takes place, usually by someone else sharing my contact info (on purpose or not). People don't read the EULAs, and really, harvesting your email is a business model, and google and the others that provide email services are enriched much much more when you let them read the emails as well to better target ads to you that will not be stopped by their advertising revenue supported unsolicited advertising filter.
I also understand and appreciate and experience myself... that sometimes one doesn't have the time, nor the inclination, to do stuff like this. I'd never admonish anyone that doesn't have the time to put up with BS, but I also think that the BS comes indifferent sizes and grows over time depending on choices. I'd rather receive email due to my own operation mistakes than to get them by design from a company funded almost entirely by ad revenue, and further expect them to not read those emails to then present ads to you... outside of emails.
Have you looked at the gmail past order tracking? Have you tried to delete stuff in it? One at a time. Some people i know have years and years of amazon orders and ebay orders and shipping info and google tracks it all, because their spam filters do that, too.
To me, the inconvenience of not using gmail seems to spiritually or metaphysically outweigh the challenges of having to check a few different accounts I set up myself on a domain or two that I control, but I am weird like that I guess. I am lazy, don't get me wrong, but I'll put in as much effort as I can to ensure that I *can* be lazy.
That said, gmail is a great solution for people that don't share my views, in whole or in part, and I don't hold it against them--like I said, not everyone cares, and sometimes, what they do care about is using that time otherwise spent on server stuff, and using it on family or life or work or anything more interesting. I can't argue with what makes a priority a priority, but I for one don't want some tech company finding out what my priorities are so that they and their valued third party business affiliates with seperate privacy policies and security policies can better advertise to me about these priorities of mine that I didn't share with them to begin with.
sorry if this was disjointed; I couldn't write it all at once and so this may not appear fluid or cohesive.. but I think the point is made. Also, Grishnakh, this isn't an attack in any way... please don't take it like one. My opinion is sort of strong but I am biased in that in both work and outside of work, I approach internet use the same way, but get paid/rewarded for it as well as a job choice. At least I have no affiliate links to send to you in email!)
(Score: 0) by Anonymous Coward on Monday September 02 2019, @02:41PM
I don't need Gmail. I'm pretty sure I can totally screw up IMAP all by my self.
(Score: 5, Informative) by nobu_the_bard on Tuesday September 03 2019, @01:24PM
Yeah he doesn't address the hardest parts in my opinion.
* Keeping your systems up to date. This is admittedly more of an issue with someone like me that has to run many mail systems besides doing many other things. Mail systems have a lot of moving parts. Changing out some parts (updating) sometimes causes problems in other parts, and merely running apt-get update or whatever does not necessarily update things like what ciphers you are using. You need to be reviewing what updates will do before you run them, anticipate what will break and handle those things, and then also handle what actually breaks when you try the update.
* Dealing with mail systems that are not correctly configured. Tons of small scale mail systems are not configured correctly. Example: You can set up to block mails conditionally based on a domain's SPF record, but you will quickly find all kinds of places violate their own screwed up SPF records all of the time (banks, marketers, etc) and you will not be able to make everyone in the world learn to fix their own things, nor can you simply block out every goofball that has a mess of a mail system, so you need to learn how to compromise on this sort of thing. Then there's other examples of goofed up systems: people running ancient MTAs with only ciphers from the 90s, people sending mails with bizarre formatting, systems that send huge amounts of junk mail but trickles of critically important mail...
I could go on, that's adequate for now.
(Score: 5, Insightful) by JoeMerchant on Monday September 02 2019, @12:28PM (4 children)
Back in 1996, e-mail clients sucked.
I bought a book on SMTP and determined that, with about 1-2 man-years of development, we could put out a "killer" e-mail app far better than anything then available.
Attachment handling hadn't completely fragmented yet, web-mail wasn't a thing, etc. Some different dot-com money floated our way, so I left the e-mail client idea on the shelf until Eudora came out with 95% of the functionality I had envisioned.
E-mail isn't hard. Secure interoperability with 7 billion actors, that's hard.
🌻🌻 [google.com]
(Score: 5, Interesting) by canopic jug on Monday September 02 2019, @01:31PM (2 children)
Back in 1996, e-mail clients sucked.
Not compared to today's variety of web mail and M$ Outlook toys. Back then, there were Pine, Mutt, and Eudora. All three blew away the capabilities of most current day clients, especially web clients. Actually, Mutt is still around now and has a quite a dedicated following because it is so darn useful. Pine, now Alpine, would be prominent too but for GMail's intentionally broken IMAP implementation which trips up Alpine. Thunderbird is new but has been rather stagnant because of Mozilla trying to kill it off again and again. However, it too blows away M$ Outlook's work flow (in regards purely to mail) and capabilities. It would be the main replacement for Eudora. And that's not bringing in late, great Procmail or its current equivalent Maildrop.
Mostly what has changed is that no new people have learned to actually use mail clients. The few times I've made the effort to look and the fewer times I've dared to interview anyone about their (lack of) e-mail skills I've been shocked with the ineptitude. Most people will assert that they know how to use e-mail but these days as good as no one actually does or maybe even actually can. It's just as bad among the M$ resellers posing as IT staff, though their fault lies with their severe affliction of the Dunning-Kruger effect dosed with some massive professional dishonesty.
Sadly the one obvious possible solution doesn't work any more. That would be to have schools, from K-12 through graduate schools and vocational schools actually get back to periodically orienting students in proper use of mail clients, mailing lists, and nettiquette. However, as shown from the fiascos that the "Internet Driving License" type trainings are, that won't get off the ground. The curriculum will not be allowed to cover anything of pith. Further, none of the teachers know how to use e-mail themselves and have basically cargo culted a few patterns of behavior that are the minimum needed to give the appearance of getting by and that is hidden in part by the crap mail tools passed off as mail clients these days.
I don't know of a way out besides what was proposed in the blog post. Or failing that, at least trying to use mail services from a smaller provider. The larger companies, that the blog author referred to as "Big Mailer Corps" are doing what they can to break mail in whatever little ways they can with an eye towards capturing or the protocols.
Money is not free speech. Elections should not be auctions.
(Score: 3, Interesting) by JoeMerchant on Monday September 02 2019, @01:42PM
Believe it or not, my 9th grader is enrolled in "digital tech" which is a pre-req for all other computer courses in high school. The syllabus purports instruction in IP addressing, DNS, e-mail, cyber safety, and of course a word from our sponsors: MS Office application use. I was a little surprised at the amount of low level stuff they say they're going to teach to "everyone" - of course, that's like learning the periodic table in Chemistry - doesn't make everyone an organic chem expert.
🌻🌻 [google.com]
(Score: 2) by SomeGuy on Monday September 02 2019, @02:26PM
This is because certain large companies want to teach everyone to "text", locking them in to buying cell phones from them.
It is also disappointing to see how few people can even write a single, coherent, paragraph exceeding 140 characters. I constantly run in to idiots that have no clue how to properly post on a forum. (Hint: a dozen one sentence posts one after the other are considered SPAM!). I've already run in to consumertards who think everyone must have a stupid cell phone with the ability to "text", with texting, Twatter, and Facefook as the "only" possible way to communicate.
(Score: 4, Insightful) by canopic jug on Monday September 02 2019, @01:34PM
Attachment handling hadn't completely fragmented yet, ...
Attachments are not a substitute for prpper file sharing. M$ killed Netware directly and, indirectly, AFS, leaving us with no replacements. Megaupload, Dropbox, Box, Spideroak are all inefficient, clunky toys. However, I'll spare that rant.
Money is not free speech. Elections should not be auctions.
(Score: 1, Interesting) by Anonymous Coward on Monday September 02 2019, @01:44PM (15 children)
Setting up an email server: not hard (excludes sendmail -- that *is* hard to impossible)
Setting up DMARC/DKIM/SPF: not hard
Running the email server: not hard (ignoring residential ISP's that block outbound port 25 -- get a different ISP in that case)
What the article, and most arguments on this topic, missed is that the setup, the configuration, and the running are not hard today (with Postfix and the other newer servers).
The part that is hard, and that which keeps most away, is learning the necessary knowledge needed to be able to setup and configure the email server. But even learning that knowledge is not hard, if one wants to learn.
But since far too many people today do not ever want to learn anything anytime, and instead want everything spoon fed to them by others, they simply parrot the meme of "email is to hard, don't do it yourself". Why? Because they define as "hard" anything that includes "learning" something.
(Score: 2) by PiMuNu on Monday September 02 2019, @02:47PM (1 child)
> But since far too many people today do not ever want to learn anything anytime
When has that ever not been the case.
(Score: 0) by Anonymous Coward on Tuesday September 03 2019, @04:05PM
As I said downthread:. that's *everyone*. I learn new tech stuff all of the time. But it would be handy for me to learn cooking, foreign languages, car repair, plumbing, carpentry, etc... And I don't. Most people, even really bright ones, learn some things in a few categories of interest and not in any others.
(Score: 4, Insightful) by Pino P on Monday September 02 2019, @03:14PM (11 children)
Does the advice to "get a different ISP" include moving yourself, your SO, and your children to a different city for the primary purpose of ending up in the service area of an ISP that is friendly to home-based side businesses and other power users? And if so, how will you afford a second move in case your ISP ends up acquired by one that imposes a policy less friendly to power users?
(Score: 1, Informative) by Anonymous Coward on Monday September 02 2019, @06:16PM (10 children)
Most ISP's offer 'business' class lines for a bit more per month.
Two of the advantages of business lines often are:
So there's no moving necessary, just "get a different ISP" (i.e., get a business class line, which is often with a 'different' ISP because the business and residential sides of the company named X are often operated independently).
(Score: 2) by RS3 on Tuesday September 03 2019, @03:29PM (8 children)
"get a different ISP" does not exist, and won't until we have true competition. My options are bad and worse: Comcast and Verizon- dumb and dumber.
Of course they play games with prices, but generally "business class" costs more than double the residential rate.
Your statement about "no blocked ports" is somewhere between fantasy and complete fabrication. Not sure where you live or what options are available to you, but here in the US, with one of the, if not the biggest, Verizon, port 25 is completely blocked. For businesses SMTP is on port 587, and for residential, 465.
As I commented elsewhere in this discussion, Verizon will only pass emails with a valid verizon.net email in the "from" field.
The best fix I know of: the wires (fibers) need to be owned publicly, then the ISP can be anyone anywhere, and they can compete reasonably.
(Score: 0) by Anonymous Coward on Tuesday September 03 2019, @04:56PM
exactly. you don't need port 25. port 25 is for losers.
(Score: -1, Troll) by Anonymous Coward on Tuesday September 03 2019, @05:24PM (4 children)
Spoken by someone who clearly has no working knowledge.
I've got Verizon FIOS, business class, and port 25 is not blocked, for either direction. The difference is business class. In fact, no TCP or UDP ports are blocked. That's what gets you the "unblocked ports" link in today's world, the magic "business class" link.
How do I know port 25 is not blocked. Because I've been running my own mail server in my basement on the FIOS link for my domain for the numerous years I've had the link now, sending and receiving emails just fine over Verizon's network.
(Score: 1) by DECbot on Tuesday September 03 2019, @09:12PM (3 children)
Back when Verizon FIOS was an option, I had a server working just fine on port 25 with the residential service. Though I suspect this have likely changed in the last 10 years.
cats~$ sudo chown -R us /home/base
(Score: 2) by RS3 on Wednesday September 04 2019, @01:06PM (2 children)
> Back when Verizon FIOS was an option...
Interesting. It's not anymore? Maybe you moved...
The little hosting company I took over as admin for about 11 years ago was supplied by a T1, and a Comcast line. They had over 512 static IPs! Very few actually used / assigned. No clue who did what or why- that's what I inherited.
Owner wanted / needed to reduce costs, and FIOS was available, so I integrated everything into 5 static IPs on business FIOS (we had an option for 5 more if needed). Port 25 used to work perfectly, but as I wrote elsewhere in this discussion, Verizon slowly but surely chipped away at it, first moving to port 587 which required an fairly easy authentication mechanism, but which 100% broke stupid Qmail (idiot code- that project needed to die).
But then more and more limitations. Maybe we're blacklisted because some of the clients' websites were being used to send spam through the webform, but it was very minimal, and Verizon have very effective spam scanning / filtering, so I'm not sure what all the whining is about.
Even at home I'm on Verizon (sometimes- I also have a Comcast Xfinity account login and can get neighbor's WiFi- completely legal- it's part of Xfinity) and they completely shut off port 25- sending is on port 465.
(Score: 2) by Pino P on Wednesday September 04 2019, @02:56PM
Verizon sold many of its landline service areas to Frontier Communications, including where I live. Subscribers were switched from Verizon FiOS to Frontier FiOS.
(Score: 1) by DECbot on Wednesday September 04 2019, @03:45PM
I was in a Verizon FIOS area in Virginia until moving cross country. Frontier FIOS in Indiana port 25 was open, but then I moved again and I am now limited to Comcast or one bar of cellular.
cats~$ sudo chown -R us /home/base
(Score: 2) by Pino P on Wednesday September 04 2019, @03:01PM (1 child)
AC meant that, for example, Xfinity (Comcast's home offering) and Comcast Business (Comcast's business offering) are technically different ISPs with different policies that happen to share a parent company.
(Score: 2) by RS3 on Thursday September 05 2019, @12:21AM
Yeah, I get that, but thanks for clearing it up for others who might get some useful info here. I think Verizon is the same way, or became that when they bought AOL and Yahoo!.
Somewhere in this discussion I mention that I have Verizon residential at home, but am part-time admin for a small hosting company that connects through Verizon Business. Verizon Home uses port 465 for smtp, but Verizon Business used port 587 for sending.
If you're a true nerd, or just feeling masochistic: https://pepipost.com/blog/smtp-port-465/ [pepipost.com]
(Score: 2) by Pino P on Wednesday September 04 2019, @02:58PM
A user of the green site claims [slashdot.org] that business ISPs in one country have a standard practice of refusing service to individuals and putting even businesses behind NAT until they lease static IPs at an additional monthly fee. So in addition to switching from residential to business service, one has to price out the fees to form an LLC and to add a static IP address, and the price quickly becomes prohibitive for a residential power user.
(Score: 1, Interesting) by Anonymous Coward on Tuesday September 03 2019, @01:09AM
I think that attitude is unfair. Most people, including tech industry professionals, don't like to learn new things when they aren't forced to do it. I could save a good bit of money making my own lattes, my own beer, my own Mexican food, my own Thai food, my own Indian food. I could save a good bit of money doing my own plumbing repairs, and electrical wiring upgrades, and auto maintenance and repairs. I could save money and open up the possible options for Halloween outfits for my kids if I learned sewing and related arts and crafts. I'd have an easier time vacationing in Mexico if I learned Spanish and a nicer time vacationing in Japan if I learned Japanese. My dogs would require less effort to take care of and have a better quality of life if I learned how to train them better. My life would be easier if I could do furniture repair (since I have kids and dogs).
I host my own email. So what. I learned how to use Docker and AWS. So what. But generally I'm one of those too many people that do not ever want to learn anything, anytime - I'm proficient at very few of the skills on that long list above. And unless you're a Renaissance Man (or Renaissance Woman or Person or whatever) - I doubt you're proficient in many of them either.
(Score: 5, Informative) by All Your Lawn Are Belong To Us on Monday September 02 2019, @03:21PM
1) E-mail isn't hard if you configure correctly = E-mail isn't hard if you know how to do it right = Easy things are easy if you know how to do them. (Unspecified: the effort required to learn how to do it right).
2) " most operating systems and distributions provide multiple alternatives, pre-packaged so you can install and run with a simple command." = "Rely on what's already there." = Using a store bought hammer is better than forging your own. (But was anybody saying email is hard because you should be forging your own hammer / writing your own packages by scratch?)
3) Industrial spammers trying to get a relay are looking to use misconfigured protocols. See Point 1.
4) "You will never reach “absolute 0 spam”, it was proven mathematically in the 2000s, but the amount you’ll receive while self-hosted can be as low or lower as what you receive at Big Mailer Corps." But unsaid is that Big Mailer Corps filter spam by leveraging their economy of scale. It doesn't matter if the amount is same if I can't filter it as efficiently as the big boys, or have to spend more time developing my filter rules than I do in reading/replying to my legitimate email.
5) Sending to enterprise players who have rules is not hard if you know and abide their rules. This one likely has some truth to it in terms of not a tremendous amount to learn and implement. It still has a flavor, though, of "take the time to learn it and it is easy." Even author admits this, "This is why I wrote this has more substance than the other claims. It’s true that IF you don’t even try to do the minimum work, THEN you’ll start with a penalty."
The whole thing amounts to "it's not hard if you take the time to learn it," while conveniently ignoring that the real arguments are that the economy of scale of big players makes the amount they charge for services less than the value I place on my time to do other things with it. If someone wants to run their own email server as a hobby, that's cool, if their ISP is cool with it. Just like running anything else as a server service: if you want to spend the time to make it work, more power to you. But trying to suggest that I'll end up better off by replicating what others do more cheaply doesn't sound like a solution. And if this person has reached the level of thinking of opening his own commercial mailing service then his level of knowledge is such that I question whether he remembers what it was like to be starting out in it. I also wonder how many hours he has spent getting to the point where he now finds it easy.
I could be off base, and I know there are Soylentils who run their own mail servers here and that's cool. It could be legitimate but this reads a little like a Meteorologist saying, "Understanding the weather is easy.... if you understand the weather."
This sig for rent.
(Score: 3, Informative) by Pino P on Monday September 02 2019, @03:25PM
In one breath, the featured article claims forward-confirmed reverse DNS, SPF, and DKIM are enough to reach the users of another mail server. But in another, the article recommends setting up an IP reputation system such that these three aren't enough:
How does one build IP reputation in the first place without enlisting the services of a Big Mailer Corp?
And how does a subscriber to a home ISP set up the forward-confirmed reverse DNS anyway without paying beaucoup bucks per month to upgrade to a static IP? I've read reports of there being cities (or entire countries such as Myanmar) where all subscribers start behind carrier-grade NAT, only business subscribers may lease a static IP at confiscatory extra monthly cost, and upgrading to business service requires business paperwork, such as articles of incorporation. Putting mail on a VPS doesn't count either because AWS and other VPS providers are also Big Mailer Corps.
(Score: 4, Insightful) by PiMuNu on Monday September 02 2019, @05:25PM (4 children)
... the problem is the whole stack.
* Ever tried to thread emails?
* Ever tried to chat to someone over email?
* Ever tried to recover an email you sent a couple of months back?
* Ever tried to send a big photo?
* Ever tried to access email on a mobile device?
I know, IRC is for chat; FTP is for sending big photos (or http nowadays or whatever); thunderbird can filter/search emails reasonably well, etc etc.
BUT the point is, this is *basic stuff* that WhatsApp and FriendFace and all the rest can do without faffing with multiple clients and generally soaking up time in worthless endeavour. So *email is deprecated* for the great majority of people.
(Score: 2, Insightful) by Anonymous Coward on Monday September 02 2019, @06:12PM (3 children)
Mutt threads emails just fine. Granted, email threading is not as accurate as Usenet threading, due to lack (or non-use) of the References header by a lot of clients, but otherwise mutt does a fine job threading emails.
UI issue, not protocol/transport issue. There is nothing stopping an email client from providing a chat UI interface for an email thread (beyond the fact that no one has written one yet).
Ever tried to recover an IM you sent a few seconds back?
Non-permanence (or lack of local storage of 'sent' items) is another UI issue, not a protocol/transport issue.
Email the transport has no restrictions on the size of an email. All size restrictions are server size restrictions, often pointless today as modern servers do not do stupid things like try to receive an entire email into RAM before writing it to disk (which is where the original size restrictions imposed by servers came from, to prevent resource exhaustion).
K9-Mail on a mobile device accesses email just fine (and it even threads). Again, UI issue, not transport/protocol issue.
(Score: 2) by nobu_the_bard on Tuesday September 03 2019, @01:02PM (2 children)
Email size restrictions are hardly pointless. For example: there's a lot of issues with multiplicative effects for example.
Consider: If I send a 100 MB email to 100 users on your personal hosted mail platform, it can explode out into 10,000 MB (10 GB). The system suddenly is dumping 10 GB of data out of nowhere into users' mailboxes, which depending on how users are configured can cause all kinds of trouble. As you can imagine this problem only gets worse the larger the attachment sizes you allow.
There's other problems like the effect on spam scanning. Also, larger mails take longer to process, and users are accustomed to email being (seemingly) instantaneous.
(Score: 0) by Anonymous Coward on Tuesday September 03 2019, @01:53PM (1 child)
The point being that the restriction is simply artificial, for reasons other than the email transport protocol.
(Score: 2) by PiMuNu on Friday September 06 2019, @01:00PM
Sure, there are workarounds, fixes, improvements. My argument was all about what is implemented and available *now*. I understand that all of this stuff is available *now* through WhatsApp or Slack; the only issue is walled gardens, which for the vast majority of users is not an issue.
(Score: 3, Informative) by ilsa on Tuesday September 03 2019, @08:23PM
I am impressed by how often this seems to be coming up lately. What frustrates me is how much people downplay the spam aspect.
No, running an SMTP server is not hard. But neither is playing a game of Go.
But like playing Go, it's not the base rules that are difficult, it's the innumerable levels of bullshit you have to deal with in the process. Setting up a mail server that bother operates as a good 'net citizen, and properly handles spam, and doesn't allow your mail to get caught in other people's spam filters, is an absolute, complete pain in the ass.
You *absolutely* need to make sure several critical pieces are in place, some of which may not even be possible to do depending on how irritating your ISP is.
-Your EHLO has to be exactly right
-You need a Reverse pointer address configured
-You need an SPF record
-You need DKIM set up
-make absolutely certain as to your server settings that you haven't created an open relay.
And even if you do everything correct, you _still_ might find your mail not reaching it's destination because the destination server may be using some kind of half-assed configuration or spam protection service that insists your email follow some non-RFC compliant thingamabob that you couldn't possibly have anticipated ahead of time.
And this is just for sending mail. I haven't gotten into how much of a pain dealing with incoming email/spam is.
The fact is, the good guys lost. They lost a long time ago. It is so easy to set up a mass-spamming operation that it's basically playing a game of whackamole. Personal anti-spam tools are just not good enough anymore, because the spam landscape is constantly changing. Your only option is to find a spam service that _proactively_ monitors incoming spam, implements honeypots, RBLs, etc, and updates it's rules constantly. Anyone that relies solely on basic Bayesian rule filtering is going to have a hard time. You'll probably want to implement RBLs on your server as well.
MXLogic used to be an excellent service, but then Intel bought them for god-knows-why reasons, and then sank the whole ship. We've been struggling to find a decent anti-spam service ever since.
A lot of these things are set-once-and-forget, but spam has taken what was once a very useful service and turned it into a Mad Max hellscape that requires constant monitoring, tweaking, etc. If you are an end-user, it's annoying. As a sysadmin, it's a bloody nightmare because you have users bothering you about spam, interrupting whatever real work you're trying to do.
So no, technically running a mail server is not hard. It's just an effing pain in the ass, and depending on what you're trying to do, may simply not be worth the ongoing time commitment.