After self-hosting my email for twenty-three years I have thrown in the towel. The oligopoly has won.
Many companies have been trying to disrupt email by making it proprietary. So far, they have failed. Email keeps being an open protocol. Hurray?
No hurray. Email is not distributed anymore. You just cannot create another first-class node of this network.
Email is now an oligopoly, a service gatekept by a few big companies which does not follow the principles of net neutrality.
I have been self-hosting my email since I got my first broadband connection at home in 1999. I absolutely loved having a personal web+email server at home, paid extra for a static IP and a real router so people could connect from the outside. I felt like a first-class citizen of the Internet and I learned so much.
Over time I realized that residential IP blocks were banned on most servers. I moved my email server to a VPS. No luck. I quickly understood that self-hosting email was a lost cause. Nevertheless, I have been fighting back out of pure spite, obstinacy, and activism. In other words, because it was the right thing to do.
But my emails are just not delivered anymore. I might as well not have an email server.
So, starting today, the MX records of my personal domain no longer point to the IP of my personal server. They now point to one of the Big Email Providers.
I lost. We lost. One cannot reliably deploy independent email servers.
This is unethical, discriminatory and uncompetitive.
*Record scratch*
*Freeze frame*
Wait, uncompetitive?
Please bear with me. We will be there in a minute.
First, some basics for people who may not be familiar with the issue.
This doesn't only affect contrarian nerds
No need to trust my word. Google has half a billion results for "my email goes directly to spam". Search any technical forum on the internet and you will find plenty of legitimate people complaining that their emails are not delivered.
What's the usual answer from experienced sysadmins? "Stop self-hosting your email and pay [provider]."
Having to pay Big Tech to ensure deliverability is unfair, especially since lots of sites self-host their emails for multiple reasons; one of which is cost.
Newsletters from my alumni organization go to spam. Medical appointments from my doctor who has a self-hosted server with a patient intranet go to spam. Important withdrawal alerts from my bank go to spam. Purchase receipts from e-commerces go to spam. Email notifications to users of my company's SaaS go to spam.
You can no longer set up postfix to manage transactional emails for your business. The emails just go to spam or disappear.
One strike and you're out. For the rest of your life.
Hey, I understand spam is a thing. I've managed an email server for twenty-three years. My spamassassin database contains almost one hundred thousand entries.
Everybody receives hundreds of spam emails per day. Fortunately, email servers run bayesian filtering algorithms which protect you and most spam doesn't reach your inbox.
Unfortunately, the computing power required to filter millions of emails per minute is huge. That's why the email industry has chosen a shortcut to reduce that cost.
The shortcut is to avoid processing some email altogether.
Selected email does not either get bounced nor go to spam. That would need processing, which costs money.
Selected email is deleted as it is received. This is called blackholing or hellbanning.
Which email is selected, though?
Who knows?
Big email servers permanently blacklist whole IP blocks and delete their emails without processing or without notice. Some of those blacklists are public, some are not.
When you investigate the issue they give you instructions with false hopes to fix deliverability. "Do as you're told and everything will be fine".
It will not.
I implemented all the acronyms1, secured antispam measures, verified my domain, made sure my server is neither breached nor used to relay actual spam, added new servers with supposedly clean IPs from reputable providers, tried all the silver bullets recommended by Hacker News, used kafkaesque request forms to prove legitimity, contacted the admins of some blacklists.
Please believe me. My current email server IP has been managed by me and used exclusively for my personal email with zero spam, zero, for the last ten years.
Nothing worked.
Maybe ten years of legitimate usage are not enough to establish a reputation?
My online community SDF was founded in 1987, four years before Tim Berners Lee invented the web. They are so old that their FAQ still refers to email as "Arpanet email". Guess what? Emails from SDF don't reach Big Tech servers. I'm positive that the beards of their admins are grayer than mine and they will have tried to tweak every nook and cranny available.
What are we left with?
You cannot set up a home email server.
You cannot set it up on a VPS.
You cannot set it up on your own datacenter.
At some point your IP range is bound to be banned, either by one asshole IP neighbor sending spam, one of your users being pwned, due to arbitrary reasons, by mistake, it doesn't matter. It's not if, it's when. Say goodbye to your email. Game over. No recourse.
The era of distributed, independent email servers is over.
Email deliverability is deliberately nerfed by Big Tech
Deliberately?
Yes. I think we (they) can do better, but we (they) have decided not to.
Hellbanning everybody except for other big email providers is lazy and conveniently dishonest. It uses spam as a scapegoat to nerf deliverability and stifle competition.
Nowadays, if you want to build services on top of email, you have to pay an email sending API which has been blessed by others in the industry. One of them.
This concept may sound familiar to you. It's called a racket.
It's only a matter of time that regulators realize that internet email is a for-profit oligopoly. And we should avoid that.2
The industry must self-establish clear rules which are harsh on spammers but give everybody a fair chance.
A simple proposal where everybody wins
Again, I understand spam is a problem which cannot be ignored. But let's do better.
We already have the technology in place but the industry has no incentives to move in this direction. Nobody is making a great fuss when small servers are being discriminated against, so they don't care.
But I believe the risk of facing external regulation should be a big enough incentive.
I'm not asking for a revolution. Please hear my simple proposal out:
- Let's keep antispam measures. Of course. Continue using filters and crowdsourced/AI signals to reinforce the outputs of those algorithms.
- Change blacklisting protocols so they are not permanent and use an exponential cooldown penalty. After spam is detected from an IP, it should be banned for, say, ten minutes. Then, a day. A week. A month, and so on. This discourages spammers from reusing IPs after the ban is lifted and will allow the IP pool to be cleaned over time by legitimate owners.
- Blacklists should not include whole IP blocks. I am not responsible for what my IP neighbor is doing with their server.
- Stop blackholing. No need to bounce every email, which adds overhead, but please send a daily notification to postmaster alerting them.
- There should be a recourse for legitimate servers. I'm not asking for a blank check. I don't mind doing some paperwork or paying a fee to prove I'm legit. Spammers will not do that, and if they do, they will get blacklisted anyways after sending more spam.
These changes are very minor, they mostly keep the status quo, and have almost no cost. Except for the last item, all the others require no human overhead and can be implemented by just tweaking the current policies and algorithms.
Email discrimination is not only unethical; it's a risk for the industry
Big Tech companies are under serious scrutiny and being asked to provide interoperability between closed silos such as instant messaging and social networks.
Well, email usage is fifteen points above social networking.
Talk about missing the forest for the trees. Nobody noticed the irony of regulating things that matter less than email.
Right now institutions don't talk about regulating email simply because they take it for granted, but it's not.
In many countries politicians are forced to deploy their own email servers for security and confidentiality reasons. We only need one politician's emails not delivered due to poorly implemented or arbitrary hellbans and this will be a hot button issue.
We are all experiencing what happened when politicians regulated the web. I hope you are enjoying your cookie modals; browsing the web in 2022 is an absolute hell.
What would they do with email?
The industry should fix email interoperability before politicians do. We will all win.
[1] I didn't clarify this at first because I didn't want this article to turn into an instruction manual. This is what I implemented: DKIM, DMARC, SPF, reverse DNS lookup, SSL in transport, PTR record. I enrolled on Microsoft's JMRP and SNDS, Google postmaster tools. I verified my domain. I got 10/10 on mail-tester.com. Thanks to everybody who wrote suggesting solutions, but I did not have a configuration issue. My emails were not delivered due to blacklists, either public or private. Back
[2] Hey, I get it. Surely my little conspiracy theory is exaggerated. Some guy on Hacker News will tell me that they work as a SRE on Gmail and that I'm super wrong and that there are 100% legit reasons as to why things are this way. Okay. Do something for me, will you? Please unread this last section, I retract it. I just needed to get it out of my system. Thanks for indulging me. Done? Good. Everything else above is a fact. Email in 2022 is anti-competitive. The Gmail guy can go explain himself to the US Senate or the European Commission. Back
Mass cellphone surveillance experiment in Spain
Spanish Statistics Institute will track all cellphones for eight days (2 min, link in Spanish, via)
A few facts first:
- Carriers geotrack all users by default, using cell tower triangulation. They also store logs of your calls and sms, but that is a story for another day.
- This data is anonymized and sold to third parties constantly, it's part of the carriers business model
- With a court order, this data can be used to identify and track an individual...
- ... which means that it is stored de-anonymized in the carrier servers
- This has nothing to do with Facebook, Google or Apple tracking with cookies or apps
- You cannot disable it with software, it is done at a hardware level. If you have any kind of phone, even a dumbphone, you are being tracked
- It is unclear whether enabling airplane mode stops this tracking. The only way to make sure is to remove the SIM card and battery from the phone.
This is news because it's not a business deal but rather a collaboration between Spain's National Statistics Institute and all Spanish carriers, and because it's run at a large scale. But, as I said above, this is not technically novel.
On paper, and also thinking as a scientist, it sounds very interesting. The actual experiment consists on tracking most Spanish phones for eight days in order to learn about holiday trips. With the results, the Government expects to improve public services and infrastructures during holiday season.
The agreement indicates that no personally identifiable data will be transferred to the INE, and I truly believe that. There is nothing wrong about using aggregated data to improve public services per se, but I am concerned about two things.
First of all, Spain is a country where Congress passed a law to create political profiles of citizens by scraping social networks —fortunately rejected by the Supreme Court— and also blocked the entire IPFS gateway to silence political dissent.
I'd say it is quite reasonable to be a bit suspicious of the use that the Institutions will make of our data. This is just a first warning for Spanish citizens: if there is no strong backlash, the next experiment will maybe work with some personal identifiable data, "just to improve the accuracy of results". And yada yada yada, slippery slope, we end up tracking individuals in the open.
Second, and most important. This is no longer a topic of debate! We reached a compromise a few years ago, and the key word is consent.
All scientists have to obtain an informed and specific consent to work with personal data, even if it is anonymous, because it is trivially easy to de-anonymize individuals when you cross-reference the anonymous data with known data: credit cards, public cameras, public check-ins, etc. In this case, once again, the Spanish institutions are above the law, and also above what is ethically correct.
No consent, no data shared, end of story. Nobody consented to this nor were we given an option to opt out.
P.S. Of course, this is a breach of GDPR, but nobody cares.
US Software companies comply with international law, to their great regret
This week has been very heavy on China-related software scandals:
- Adobe deletes accounts from Venezuela due to US exports law (1 min, via)
- After a 'South Park' Censorship Episode, China Deleted the Entire Show From the Web (5 min, via)
- Blizzard bans Hearthstone Grandmasters winner and forfeits their prize due to Hong Kong statements (1 min, via)
On Apple's side, as usual, ther has been more media coverage:
- Apple Hides Taiwan Flag in Hong Kong (2 min, via)
- China state newspaper criticizes Apple for app use by Hong Kong protesters (1 min, via)
- Apple Removes HKmap.live from the App Store (1 min, via), and
- Tim Cook write a company-wide memo with the Company's official stance (1 min)
US companies and entities are forced to apply international law, sometimes breaking universal human rights.
This is a difficult topic. On one hand, States are sovereign. On the other, we should push for a better world. However, to which degree has a private company the right to ignore state rulings? They can, and suffer the consequences. That would be consistent. Are they ready to boycott a whole country, or risk a banishment from that country?
As an individual, the take home message is that if you delegate some of your tasks to a private company, or you rely on a private company in some degree, you risk being unable to access your data or virtual possessions at any time. Be it due to international law, or to some stupid enforcement or terms-of-service bullshit.
Please follow the HN discussions on the "via" links above, they are very informative.
Living in a disrupted economy
There is this continuing discussion on whether technology destroys more jobs than it creates. Every few years, yet another tech revolution occurs, journalists publish articles, pundits share their opinions, politicians try to catch up, and those affected always voice their concerns. These couple years have been no exception, thanks to Uber, Airbnb, and the called sharing economy.
I'm a technologist and a relatively young person, so I am naturally biased towards technological disruption. After all, it is people like me who are trying to make a living by taking over older jobs.
I suggest that you take a few minutes to read a fantastic article titled The $3500 shirt. That essay reveals how horrible some industries were before they could be automated or replaced by something better. Go on, please read it now, it will only take three minutes.
Now, imagine you had to spend a couple of weeks of your time to make a t-shirt from scratch. Would that be acceptable? I guess we all more or less agree that the textile revolution was a net gain for society. Nevertheless, when it occurred, some Luddites probably complained, arguing that the loom put seamstresses out of work.
History is packed with dead industries. We killed the ice business with the modern fridge. We burn less coal for energy, so miners go unemployed. And let's not forget the basis of modern civilization, the agricultural revolution, which is the only reason us humans can feed ourselves. Without greenhouses, nitrates, tractors, pest protection and advancements in farming, humanity would starve.
Admittedly, it transformed the first sector from a 65% in workforce quota into the current 10%. Isn't it great that most of us don't need to wake up before sunrise to water our crops? In hindsight, can you imagine proclaiming that the 1800s way of farming is better because it preserves farming jobs?
The bottom line is that all economic transformations are a net gain for society. They may not be flawless, but they have allowed us humans to live a better life.
So why do some characters fight against current industry disruptions if history will prove them wrong?
As a European and a social democrat, I believe that States must regulate some economies to avoid monopolies and abuses, supporting the greater good. Furthermore, I sympathize with the affected workforce, both personally and in a macroeconomic level. All taxi drivers suddenly going jobless because of Uber is detrimental to society.
However, it pains me to see that European politicians are taking the opposite stance, brandishing law and tradition as excuses to hinder progress.
Laws must serve people, not the other way around. If we analyze the taxi example, we learn that there is a regulation which requires taxi drivers to pay a huge sum of money up front to operate. Therefore, letting anybody get in that business for free is unfair and breaks the rules of the game. Unsurprisingly, this situation is unfair not because of the new players, but because that regulation is obsolete.
It isn't ethically right that somebody who spent a lot of money to get a license sees their job at risk. But the solution isn't to block other players, especially when it's regulation which is at fault. Let's sit down, think how to establish a transition period, and maybe even reimburse drivers part of that money with the earnings from increased taxes due to a higher employment and economic activity.
There is a middle ground solution: don't change the rules drastically, but don't use these them as an excuse to impede progress.
At the end of the day, some careers are condemned to extinction. That is a real social drama, however, what should we do? Artificially stop innovation to save jobs which are not efficient and, when automated or improved, they make the world better for everyone?
Us millennials have learned that the concept of a single, lifetime profession just does not exist anymore. Previous generations do not want to accept that reality. I understand that reconverting an older person to a new career may be difficult, but if the alternative is letting that person obstruct younger people's opportunities, that's not fair.
Most professions decline organically, by the very nature of society and economy. It is the politicians' responsibility to mediate when this process is accelerated by a new industry or technology. New or automated trades will take their place, usually providing a bigger collective benefit, like healthcare, education, or modern farming.
Our duty as a society is to make sure everyone lives a happy and comfortable life. Artificially blocking new technologies and economic models harms everyone. If it were for some Luddites, we'd be still paying $3500 for a shirt, and that seamstress would never have been a nurse or a scientist.
"Think of the terrorists" is the new "Think of the children"
If I am prime minister, I will make sure that it is a comprehensive piece of legislation that makes sure we do not allow terrorists safe space to communicate with each other. That is the key principle: do we allow safe spaces for them to talk to each other? I say no, we don't, and we should legislate accordingly.
—David Cameron
What infuriates me the most is that is such a blind, selfish, first world argument. It implies freedom of speech is granted, ubiquitous, and irreversible, so those who want extra protection must be criminals. Mr. Cameron's statement also assumes that there is no middle ground, and all technologies that can be misused by some party should be illegal. You know, the Hitler-croquettes theorem: since Hitler liked croquettes, croquettes must be bad.
In some countries, the Government can kill you for your political views. Your neighbors can also kill you for what you are—gay, for example. Ill-named "activists" can kill you for private beliefs that don't affect other people, like your stance on abortion. Mafias can kill you for badmouthing. And these all happen in first world countries, can you imagine the rest of the world?
Requesting those people to abandon the tool that is currently saving their lives in exchange for the vague promise of finding terrorists is a false dichotomy. I can understand uneducated people considering this topic as black and white. But a Prime Minister? That's a supreme level of blindness.
Mr. Cameron and others surely understand how the world works. They know that hackings, theft, revolutions, and coups d'etat exist, and those who once were righteous, legal and legitimate may be prosecuted. Something being legal or punishable can quickly change, it is not written in stone, and definitely not universal.
Imagine a Christian in 2011 Syria. They lead an ordinary life, have a job, a Facebook, they send funny memes to their friends, they communicate online. Being a Christian something we can agree is a legitimate and harmless belief and, according to 2011 Syria's laws, legal.
Now meet ISIS. In just a year they have conquered a large portion of the territory and changed some laws considerably. Forbidding Christians in Syria to use encryption is, with Mr. Cameron's words, not allowing people a safe space to communicate with each other, and exposing them to ISIS. You see, in some cases, banning encryption helps terrorists.
That is not a paradox. Encryption is a tool, like a knife, a chainsaw or a Bic pen. Banning a tool has consequences, and arguing at a fallacy level with something as serious as the lives of people is deeply insulting.
We need encryption, period. Personal communications must be private, period. We can discuss the transparency/secrecy balance for governments, but that is a topic for another day.
Governments must find some other way of fighting crime than just exposing everybody naked to make it easier to pick the bad apples.
Encryption is saving lives of gays, Muslims, activists, individuals who are threatened. It is allowing Mr. Cameron to send private texts to their wife without The Sun intercepting them. It is what avoids ISIS to spy on the UK Ministry of Defence intelligence. Does he really not realize that? Is he not that bright? Is he ill-advised? Is he just a hypocrite?
Encryption is avoiding that in a massive wirelessly connected world anybody can listen to what everybody else is saying in any part of the planet. Do we allow safe spaces for people to talk to each other? I say yes, we do, and we should legislate accordingly
When it comes to privacy and accountability, people always demand the former for themselves and the latter for everyone else
—David Brin
Hey NSA, as you sow, so shall you reap
It looks like the new "encrypted by default" policy on smartphones is freaking out law enforcement agencies. Honestly, what were they expecting? They have been abusing laws and courts for so long that we are starting to take measures to let private companies protect us from our governments. How twisted is that, huh?
"When I see a police officer now, instead of protected, I feel threatened." That's a bit demagogic but bears some truth. People seem to have interiorized that concept and we now prefer to have some privacy, regardless of what police think. Yes, we are so busy caring for our safety that we don't give a crap if that interferes with the FBI —probably necessary— counterterrorism work.
But wait, is that true? I mean, isn't that reasoning a bit flawed? Are people stupid or careless?
When you think about it for a minute, there is a crucial point. Who is more likely to have resources to circumvent police investigations? Of course, professional criminals. That's why you can't make a backup of your DVDs, but pirates can. Professionals always find a way, it's regular citizens who have no means to protect themselves.
This is a comic I made in 2005 (click to zoom).
It says, "The EU wants to keep a record of phone calls, SMS and emails as a security measure against terrorism." Then, an Al-Qaeda terrorist who's planning to bomb the twin towers starts using carrier pigeons. Both his phone and computer are wired to the CIA, but that's of no use now.
As time told, they passed that law, and now everyone's communications are under police eyes. It's ironic, but nowadays the communications protocol which is the most protected by law is... postal mail.
In the end, it is a false sense of security. We have to give our laptop password to a random guy on an airport and let him check our email and pictures while real terrorists have a decoy encrypted partition. They can manage all the hassle, we can't, so they win.
Or better, they will carry paper documents in a briefcase. Expect next decades' spy films to stop portraying criminals as cyberpunk hackers and go back to the 50's analogic look. In the age of Apple Watches, nobody will suspect that a Casio watch hides a microfilm with the schematics for a bomb.
Spanish media just shot themselves in the foot -- or maybe in the head
In Spain we have an old proverb, La avaricia rompe el saco. Literally "greed bursts the sack"; it means that if you fill a purse with too many coins it will break and you will end up with none.
This week, the Spanish Congress passed a law with two main goals:
- Ban torrenting sites, i.e. that is link-only sites (not content hosts), which is a totally different topic.
- Make social aggregators pay media publishers for the use of news excerpts.
More details can be found on this Gizmodo article
If this weren't so serious I'd say that news lobbies pressing against the right to quote, you know, the one their business is based on, is ironic.
But this is so outrageously hypocritical that it's not ironic, it's immoral and vomitive. Disgusting. Greedy to the extreme. This is a capital crime against ethics.
So why did they just do that?
Last year, Google was forced to pay French publishers for use of their content. Spanish publisher lobby AEDE (lack of link intended) saw here a huge opportunity: let's do the same and get free money from Google.
Google is so big that's it's an easy target. Demagogy is so simple; Google is a tech giant that does fiscal engineering to avoid paying taxes and profits from our content. Yes, that's true. But Google does exactly what these publishers do: curate what others say and provide citations to strengthen and validate their job.
But then, Google's natural reaction would have been, "You don't want my traffic? Wish granted! Next time, be careful what you wish". However, AEDE had anticipated this, so with the new law content providers can't opt out by not linking to AEDE's affiliated media. F*ck off genie, we wished for infinite wishes!
It's so effortless to lobby in a corrupt and manipulated environment where politicians don't even know what a link is.
But wait, there's more.
Once you start considering the implications of having to pay to hyperlink, things get worse. A study conducted by Coalición Prointernet, a lobby against this law, states the obvious:
- It has not been proven that content aggregation limits the editor's earnings. Of course; it's the opposite, it actually drives them more traffic—300M yearly visits, according to an admin of one of those sites.
- There is no basis to establish an inalienable compensation towards media editors and, if it were any, this new legislation is not the best way to go.
- The new law reduces legal security for Spanish internet companies.
- Media aggregation is necessary and positive from a "freedom of speech" standpoint. Unavailability of aggregators can drive small publishers to extinction and leave users without an important tool to diversify their media consumption.
Please read and think about the last point again, because it is very, very important.
Let's summarize what is happening here:
Big media editors AEDE, most of which pro-government, in collusion with the corrupt Spanish politicians have managed a masterstroke which they think will:
- Get them free money
- Destroy the discoverability of smaller media competitors, usually critical with the government
- Hinder the future of Spanish internet tech business, their main competitor
- Get more exposure, since readers won't have access to media agreggation and will resort to reading just one or two outlets
In reality, what is likely to happen is:
- Google will close Google News Spain, no big problem
- Spanish media aggregators will move their business abroad and won't contribute taxes to the country
- Tech entrepreneurs will realize that Spain is a shitty country to invest money on
- Without Google, the aggregators, and thanks to the increasing user boycott to AEDE media, those editors will lose traffic and money.
This is so, so sad.
It is clear that traditional media companies are suffering because of the internet revolution and need to fight in some way. However, they are cutting their own nose to spite the face. And, in the way, they are denying others a right, not a banal one, but the right to quote, which news business is built on.
I honestly think that traditional media is absolutely necessary even today. They are the ones who report, research, discover, analyze and interpret what's happening in the world. Specially in Spain, where we don't have these modern US internet-only media companies which don't just feast on press releases but do real journalism.
This is not a cry against traditional media. People, most of all, need them. But people also need aggregators to contrast different views on news. Aggregators need media because it's impossible to talk about news without a headline and an excerpt to reveal what's going on. And media, most of all, needs aggregators and people to survive in today's world.
Now the law has been passed. Though it needs to be ratified in the Senate, it is a pantomime because the majorities are the same as in Congress and also Congress has the last word even if the Senate votes against it (take that, Montesquieu!). What will media editors do when they start losing money and realize the harm they have done to themselves, the Government, Spanish media consumers and the Spanish tech industry?
Next time you think somebody is stupid, remember that the Spanish press just got in a war with Google, Facebook and Twitter because they want them to stop linking to their content.
Crazy world we live in, huh?
Bile is not freedom of speech, but neither is censorship
Twitter—and most sites which support user comments—are filled with filth. Unfortunately, that's an indisputable rule of the internet. Some people are just unaware of the power of their words, and write comments that expose the worst of human nature, like the guy who cheered the murder of a Spanish politician
Despicable as that comment is, however, I think that police resources would be better used if going after actual criminals or corrupt politicians instead of angry teenagers. On the other hand, people should start learning that throwing bile on internet comments may be freedom of speech but also an offence of verbal threatening. Let's just start behaving like intelligent people and try neither to threaten others nor indict dumb twitter users.
My point here is that the internet was born free, and it is now turning into a "regular" business: money will pay quality of service, online media will be regulated as if hyperlinks and news excerpts were copyright infringement, and now all internet comments will be treated as if they were uttered on the real world; at the main street or on a news outlet.
That is not necessarily bad, per se. However, the people and lobbies regulating the net are the same people who think cookies are a menace to user privacy and have strong political and economical interests on putting strong leashes to users. The internet has been bypassing their establishment for about 20 years and now it's time to put an end to that.
Should the internet be different from the rest of society? It's not a rhetorical question. However, there is no easy answer. The net has created new idiosyncrasies and, if we change the rules of the game, more things need to change. However, politicians don't understand—or don't want to—this concept.
If they charge business for their QoS, then those business must be able to demand public internet providers where there are monopolies or oligopolies. If a blog can be sued for copyright infringement, so must be TVs that feed from youtube users. The list goes on.
The problem now is that it's too tempting for the establishment to regulate everything in their favor. And, given that they have the ability to do so, puts democracy to shame. That is, if there is one.
As incredible as it sounds, the tinfoil-hat people were actually right. What's more indignant, teenagers may pay large fines for hate speech while governments steal their naked pictures from their webcams and companies take our tax money to turn it against our interests.
Regrettably, for those who didn't know it, that is how the world has always worked. The internet was a small oasis that lasted a few years and gave us a taste of real freedom, for the good and the bad—Silk Road, CP, etc.
What now? I think we should support organizations that defend our rights, software that empowers us instead of relying on the bona fide of our providers, and helping develop a new internet, if that were the case.
In Spain, we have a saying "hecha la ley, hecha la trampa". It means that the cheaters will always be one step ahead of the rules. Applied to the internet, that will be a good thing for users and freedom.
The ignorant EU cookie law
It is 2014, and many webmasters still don't know that there is a new EU law which regulates cookies and other data stored in user computers. This is part of a noble effort to protect user privacy, which, well, I personally support.
Unfortunately, the actual law is technically incompetent and does nothing for user privacy while placing a lot of responsibility on webmasters and costing them a lot of time and money.
Put it another way, this law wants to protect users by forcing spoon merchants to inform clients about the chance of being harmed by a spoon while ignoring knife, drug or gun merchants. It's useless.
This uselessness and absolute ignorance of how the Internet works is costing EU webmasters a lot of time and money. In my case, I counted them, about ten hours. For large companies, it can be a lot more.
Why am I against this law? Why do I say it is useless?
It doesn't protect user privacy
Cookies are not the only way to track us. Modern methods use just Javascript (i.e. the Facebook 'like' button) and leave no data on the user's browser.
Furthermore, the browser itself can be used to uniquely identify you. Test it
What's worse, it says nothing about doing analytics with personal data, like the IP.
And how could we forget the fact that it is the governments themselves who are spying on users? How on Earth need cookies legislation if the UK and US just steal pictures from our webcams, data from our emails and information from our text messages?
Our legislators must be absolutely ignorant or absolute hypocrites to regulate cookies while governments spy on us. I'm not sure which one is worse.
Cookie management was solved 20 years ago
Let's assume that the previous reason wasn't valid. Let's assume cookies were a real menace to user privacy.
Well, this problem was solved since IE4, when browsers invented the cookie warning popup window.
Managing cookies in the browser is the best idea, for many reasons:
- It is a central tool to manage cookies.
- Display a common interface for all cookie warnings. With the current law, every webpage displays the notification with a different style and location: on top, on the bottom, on a side, on a popup. Visitors don't know where to find it.
- We should trust user software instead of website policies. What if a website was using cookies to track me? Should I trust them? Would it solve anything that they had to pay a fine if they have already stolen my data? Concerned people should use trusted browsers, and hardened open-source operating systems, if possible. To state an example, Facebook would earn more money by breaking this law and spying on us with cookies than the fine it would have to pay if they get caught.
It confuses users
I did a quick survey with some non-technical people, asking them if they had seen this "cookies notice". They said they had. I then asked them if they did understand what it meant. They didn't.
Modern UXs have overwhelmed us with notification windows, up to a point that we just click on "dismiss" without even looking at them. Well, we should read the text, but the truth is that many people don't. Instead of arguing over what should be done, let's try to avoid contributing to the too-many-notifications problem, and just solve them on the browser.
By the way, I bet that the most clicked button on IE6 was the "Accept all cookies, do not bother me again" checkbox.
It costs people a lot of pain and money
Let's imagine there are a million websites in the EU. Let's imagine every webmaster takes, in average, 8 hours to adapt each to the new cookie laws. Let's imagine the average webmaster cost is 50€/hr
This useless law has costed EU companies and individuals 400 million euros. Nice way to impulse the internet economy.
Different countries have different requirements
In the UK, it is enough to provide a notice to tell users that the website uses cookies.
About ten years ago, browser developers decided to remove the UX label that notified the user when cookies were received because they thought there was not much to show. Now we have to implement them again, on a per-site basis. Outstanding, given that the cookie is set anyway.
Bad as it is, in Spain, a website can't set cookies unless the user accepts them, either by scrolling or clicking a link. At least, well, the user is "protected" by default, even though the technical solution is harder.
Helping the community
Angry as I am right now for having wasted ten hours of my life implementing a useless law, I thought the least I could do, besides writing a rage post, was to share my solution.
You can go to Github and download the sample I prepared. It's the same code that you can see running here if you noticed the banner. It might not be the best, but at least it gives webmasters a starting idea, and no-PHP, 100% HTML+javascript routine to run all Analytics and cookie-dependent code.
To summarize my implementation, it consists of a javascript file which handles the cookie banner, sets the actual cookie when the user gives consent, and also manages some exceptions. Unlike most of the solutions I found, which only display the banner, this code does actually handle cookies.
Check it out, and please, feel free to send pull requests and discuss its issues.
Final thoughts
I think my points are quite valid, and this is actually a useless and annoying law that serves nothing and costs money. If the regulators had consulted a competent panel, they would have learned that the cookie law does not serve their noble intention of protecting users.
The root problem, again, is that our politicians don't have the slightest idea of how the internet works. These are the same guys that now must decide on the fate of the internet as we know it.
I don't know about you, but I lost all my hopes long ago.
Did my code save you any time and money? Please donate it to the EFF.