This website uses third party cookies exclusively to collect analytics data. If you continue browsing or close this notice, you will accept their use. The EU now requires all sites to display this banner which confuses users and does nothing, actually, to improve your privacy.
Read more on why this law is ignorantLearn about this website's cookiesDisallow cookies
Carlos Fenollosa

Carlos Fenollosa

Engineer, developer, entrepreneur

Carlos Fenollosa — Blog

Thoughts on science and tips for researchers who use computers

Three take aways to understand Cloudflare's apocalyptic-proportions mess

February 24, 2017 — Carlos Fenollosa

It turns out that Cloudflare's proxies have been dumping uninitialized memory that contains plain HTTPS content for an indeterminate amount of time. If you're not familiar with the topic, let me summarize it: this is the worst crypto news in the last 10 years.

As usual, I suggest you read the HN comments to understand the scandalous magnitude of the bug.

If you don't see this as a news-opening piece on TV it only confirms that journalists know nothing about tech.

How bad is it, really? Let's see

I'm finding private messages from major dating sites, full messages from a well-known chat service, online password manager data, frames from adult video sites, hotel bookings. We're talking full HTTPS requests, client IP addresses, full responses, cookies, passwords, keys, data, everything

If the bad guys didn't find the bug before Tavis, you may be on the clear. However, as usual in crypto, you must assume that any data you submitted through a Cloudflare HTTPS proxy has been compromised.

Three take aways

A first take away, crypto may be mathematically perfect but humans err and the implementations are not. Just because something is using strong crypto doesn't mean it's immune to bugs.

A second take away, MITMing the entire Internet doesn't sound so compelling when you put it that way. Sorry to be that guy, but this only confirms that the centralization of the Internet by big companies is a bad idea.

A third take away, change all your passwords. Yep. It's really that bad. Your passwords and private requests may be stored somewhere, on a proxy or on a malicious actor's servers.

Well, at least change your banking ones, important services like email, and master passwords on password managers -- you're using one, right? RIGHT?

You can't get back any personal info that got leaked but at least you can try to minimize the aftershock.

Update: here is a provisional list of affected services. Download the full list, export your password manager data into a csv file, and compare both files by using grep -f sorted_unique_cf.txt your_passwords.csv.

Afterwards, check the list of potentially affected iOS apps

Let me conclude by saying that unless you were the victim of a targeted attack it's improbable that this bug is going to affect you at all. However, that small probability is still there. Your private information may be cached somewhere or stored on a hacker's server, waiting to be organized and leaked with a flashy slogan.

I'm really sorry about the overly dramatic post, but this time it's for real.

Tags: security, internet, news

Comments? Tweet  

What do "Pro" users want?

November 16, 2016 — Carlos Fenollosa

My current machine is a 2013 i7 Macbook Air. It doesn't have the Pro label, however, It has two USB 3.0 ports, an SD slot and a Thunderbolt port. 12 hours of battery life. One of the best non-retina screens around. Judging by this week's snarky comments, it's more Pro than the 2016 Macbook Pro.

Me, I love this laptop. In fact, I love it so much that I bought it to replace an older MBA. I really hoped that Apple would keep selling the same model with a Retina screen and bumped specs.

But is it a Pro computer or not? Well, let me twist the language. I make my living with computers, so by definition it is. Let's put it another way around: I could have spent more money for a machine which has Pro in its name, but that wouldn't have improved my work output.

What is a Pro user?

So there's this big discussion on whether the Pro label means anything for Apple.

After reading dozens of reviews and blog posts, unsurprisingly, one discovers that different people have different needs. The bottom line is that a Pro user is someone who needs to get their work done and cannot tolerate much bullshit with their tools.

In my opinion, the new Macbook Pros are definitely a Pro machine, even with some valid criticisms. Apple product releases are usually followed by zesty discussions, but this time it's a bit different. It's not only angry Twitter users who are complaining; professional reviewers, engineers, and Pro users have also voiced their concerns.

I think we need to stop thinking that Apple is either stupid or malevolent. They are neither. As a public company, the metric by which their executives are evaluated is stock performance. Infuriating users for no reason only leads to decreasing sales, less benefits, and unhappy investors.

I have some theories on why Apple seems to care less about the Mac, and why many feel the need to complain.

Has the Pro market changed?

Let's be honest: for the last five years Apple probably had the best and most popular computer lineup and pricing in their history. All markets (entry, pro, portability, desktops) had fantastic machines which were totally safe to buy and recommend, at extremely affordable prices.

I've seen this myself. In Spain, as one of the poorest EU countries, Apple is not hugely popular. Macs and iPhones are super expensive, and many find it difficult to justify an Apple purchase on their <1000€ salary.

However, in the last three to five years, everybody seemed to buy a Mac, even friends of mine who swore they would never do it. They finally caved in, not because of my advice, but because their non-nerd friends recommend MBPs. And that makes sense. In a 2011 market saturated by ultraportables, Windows 8, and laptops which break every couple years, Macs were a great investment. You can even resell them after five years for 50% of their price, essentially renting them for half price.

So what happened? Right now, not only Pros are using the Macbook Pro. They're not a professional tool anymore, they're a consumer product. Apple collects usage analytics for their machines and, I suppose, makes informed decisions, like removing less used ports or not increasing storage on iPhones for a long time.

What if Apple is being fed overwhelmingly non-Pro user data for their Pro machines and, as a consequence, their decisions don't serve Pro users anymore, but rather the general public?

First, let's make a quick diversion to address the elephant in the room because, after all, I empathize with the critics.

Apple is Apple

Some assertions you can read on the Internet seem out of touch with a company which made the glaring mistake of building a machine without a floppy, released a lame mp3 player without wireless and less space than a Nomad, tried to revolutionize the world with a phone without a keyboard, and produced an oversized iPhone which is killing the laptop in the consumer market.

Apple always innovates. You can agree whether the direction is correct, but they do. They also copy, and they also steal, like every other company.

What makes them stand out is that they are bolder, dare I say, more courageous than others, to the point of having the courage to use the word courage to justify an unpopular technical decision.

They take more risks on their products. Yes, I think that the current audio jack transition could've been handled better, but they're the first "big brand" to always make such changes on their core products.

This brings us to my main gripe with the current controversy. I applaud their strategy of bringing iPhone ideas, both hardware and software, to the Mac. That is a fantastic policy. You can design a whole device around a touch screen and a secure enclave, then miniaturize it and stick it on a Macbook as a Touch Bar.

Having said that, us pros are generally conservative: we don't update our OS until versions X.1 or X.2, we need all our tools to be compatible, and we don't usually buy first-gen products, unless we self-justify our new toy as a "way to test our app experience on users who have this product".

The Great Criticism Of The 2016 Macbook Pro is mainly fueled by customers who wanted something harder, better, faster, stronger (and cheaper) and instead they got a novel consumer machine with few visible Pro improvements over the previous one and some prominent drawbacks.

Critical Pros are disappointed because they think Apple no longer cares about them. They feel they have no future using products from this company they've long invested in. Right now, there is no clear competitor to the Mac, but if it were, I'm sure many people would vote with their wallets to the other guy.

These critics aren't your typical Ballmers bashing the iPhone out of spite. They are concerned, loyal customers who have spent tens of thousands of dollars in Apple's products.

What's worse, Apple doesn't seem to understand the backlash, as shown by recent executive statements. Feeling misunderstood just infuriates people more, and there are few things as powerful as people frustrated and disappointed with the figures and institutions they respect.

Experiment, but not on my lawn

If I could ask Apple for just one thing, it would be to restrict their courage to the consumer market.

'Member the jokes about the 2008 Macbook Air? Only one port, no DVD drive?

The truth is, nobody cared because that machine was clearly not for them; it was an experiment, which if I may say so, turned out to be one of the most successful ever. Eight years later, many laptops aspire to be a Macbook Air, and the current entry Apple machine, the Macbook "One", is only an iteration on that design.

Nowadays, Apple calls the Retina MBA we had been waiting for a "Macbook Pro". That machine has a 15W CPU, only two ports—one of which is needed for charging—, good enough internals, and a great battery for light browsing which suffers on high CPU usage.

But when Apple rebrands this Air as a Pro, real pros get furious, because that machine clearly isn't for them. And this time, to add more fuel to the fire, the consumer segment gets furious too, since it's too expensive, to be exact, $400 too expensive.

By making the conscious decision of positioning this as a Pro machine both in branding and price point, Apple is sending the message that they really do consider this a Pro machine.

One unexpected outcome of this crisis

Regardless, there is one real, tangible risk for Apple.

When looking at the raw numbers, what Apple sees is this: 70% of their revenue comes from iOS devices. Thus, they prioritize around 70% of company resources to that segment. This makes sense.


Unless there is an external factor which drives iPhone sales: the availability of iPhone software, which is not controlled by Apple. This software is developed by external Pros. On Macs.

The explosion of the iOS App Store has not been a coincidence. It's the combination of many factors, one of which is a high number of developers and geeks using a Mac daily, thanks to its awesomeness and recent low prices. How many of us got into iPhone development just because Xcode was right there in our OS?

Similarly to how difficult it is to find COBOL developers because barely anyone learns it anymore, if most developers, whichever their day job is, start switching from a Mac to a PC, the interest for iOS development will dwindle quickly.

In summary, the success of the iPhone is directly linked to developer satisfaction with the Mac.

This line of reasoning is not unprecedented. In the 90s, almost all developers were using the Microsoft platform until Linux and OSX appeared. Nowadays, Microsoft is suffering heavily for their past technical decisions. Their mobile platform crashed not because the phones were bad, but because they had no software available.

Right now, Apple is safe, and Pro users will keep using Macs not only thanks to Jobs' successful walled garden strategy, but also because they are the best tools for the job.

While Pro users may not be trend-setters, they win in the long term. Linux won in the server. Apple won the smartphone race because it had already won the developer race. They made awesome laptops and those of us who were using Linux just went ahead and bought a Mac.

Apple thinks future developers will code on iPads. Maybe that's right 10 years from now. The question is, can they save this 10-year gap between current developers and future ones?

The perfect Pro machine

This Macbook Pro is a great machine and, with USB-C ports, is future proof.

Dongles and keyboards are a scapegoat. Criticisms are valid, but I feel they are unjustly directed to this specific machine instead of Apple's strategy in general. Or, at least, the tiny part that us consumers see.

Photographers want an SD slot. Developers want more RAM for their VMs. Students want lower prices. Mobile professionals want an integrated LTE chip. Roadies want more battery life. Here's my wish, different than everybody else's: I want the current Macbook Air with a Retina screen and 20 hours of battery life (10 when the CPU is peaking)

Everybody seems to be either postulating why this is not a Pro machine or criticizing the critics. And they are all right.

Unfortunately, unless given infinite resources, the perfect machine will not exist. I think the critics know that, even if many are projecting their rage on this specific machine.

A letter to Santa

Pro customers, myself included, are afraid that Apple is going to stab them on the back in a few years, and Apple is not doing anything substantial to reduce these fears.

In computing, too, perception is as important as cold, hard facts.

Macs are a great UNIX machine for developers, have a fantastic screen for multimedia Pros, get amazing build quality value for budget constrained self-employed engineers, work awesomely with audio setups thanks to almost inaudible fans, triple-A software is available, and you can even install Windows.

We have to admit that us Pros are mostly happily locked in the Apple ecosystem. When we look for alternatives, in many cases, we only see crap. And that's why we are afraid. Is it our own fault? Of course, we are all responsible for our own decisions. Does this mean we have no right to complain?

Apple, if you're listening, please do:

  1. Remember that you sell phones because there's people developing apps for them.
  2. Ask your own engineers which kind of machine they'd like to develop on. Keep making gorgeous Starbucks ornaments if you wish, but clearly split the product lines and the marketing message so all consumers feel included.
  3. Many iOS apps are developed outside the US and the current price point for your machines is too high for the rest of the world. I know we pay for taxes, but even when accounting for that, a bag of chips, an apartment, or a bike doesn't cost the same in Manhattan than in Barcelona.
  4. Keep making great hardware and innovating, but please, experiment with your consumer line, not your Pro line.
  5. Send an ACK to let us Pros recover our trust in you. Unfortunately, at this point, statements are not enough.

Thank you for reading.

Tags: hardware, apple

Comments? Tweet  

Basic iPhone security for regular people

August 18, 2016 — Carlos Fenollosa

Real life requires a balance between convenience and security. You might not be a high-profile person, but we all have personal information on our phones which can give us a headache if it falls into the wrong hands.

Here are some options you can enable to harden your iPhone in the case of theft, a targeted attack or just a curious nephew who's messing with your phone.

Even if you don't enable them all, it's always nice to know that these features exist to protect your personal information. This guide is specific for iPhones, but I suppose that most of them can be directly applied to other phones.

Password-protect your phone

Your iPhone must always have a password. Otherwise, anybody with physical access to your phone will get access to all your information: calendar, mail, pictures or *gasp* browser history.

Passwords are inconvenient. However, even a simple 4-digit code will stop casual attackers, though it is not secure against a resourceful attacker

☑ Use a password on your phone: Settings > Touch ID & Passcode

Furthermore, enable the 10-attempt limit, so that people can't brute-force your password.

☑ Erase data after 10 attempts: Settings > Touch ID & Passcode > Erase data (ON)

If your phone has Touch ID, enable it, and use a very long and complicated password to unlock your phone. You will only need to input it on boot and for a few options. It is reasonably secure and has few drawbacks for most users. Unless you have specific reasons not to do it, just go and enable Touch ID.

☑ Enable Touch ID: Settings > Touch ID & Passcode

Regarding password input, and especially if your phone doesn't have Touch ID, using a numeric keyboard is much faster than the QWERTY one. Here's a trick that will help you choose a secure numeric password which is easy to remember.

Think of a word and convert it to numbers as if you were dialing them on a phone, i.e. ABC -> 2, DEF -> 3, ..., WYZ -> 9. For example, if your password is "PASSWORD", the numeric code would be 72779673.

The iPhone will automatically detect that the password contains only numbers and will present a digital keyboard on the lock screen instead of a QWERTY one, making it super easy to remember and type while still keeping a high level of security.

☑ If you must use a numeric password, use a long one: Settings > Touch ID & Passcode

Harden your iPhone when locked

A locked phone can still leak private data. Accessing Siri, the calendar or messages from the lock screen is handy, but depending on your personal case, can give too much information to a thief or attacker.

Siri is a great source of data leaks, and I recommend that you disable it when your phone is locked. It will essentially squeal your personal info, your contacts, tasks or events. A thief can easily know everything about you or harass your family if they get a hand on a phone with Siri enabled on the lock screen.

This setting does not disable Siri completely; it just requires the phone to be unlocked for Siri to work.

☑ Disable Siri when phone is locked: Settings > Touch ID & Passcode > Siri

If you have confidential data on your calendar, you may also want to disable the "today" view which usually includes your calendar, reminders, etc.

☑ Disable Today view: Settings > Touch ID & Passcode > Today

Take a look at the other options there. You may want to turn off the notifications view, or the option to reply with a message. An attacker may spoof your identity by answering messages while the phone is locked, for example, taking advantage from an SMS from "Mom" and tricking her into asking for her maiden name, pet names, etc., which are usually answers to secret questions to recover your password.

☑ Disallow message replies when the phone is locked: Settings > Touch ID & Passcode > Reply with Message

Having your medical information on the emergency screen has pros and cons. Since I don't have any dangerous conditions, I disable it. Your case may be different.

Someone with your phone can use Medical ID to get your name and picture, which may be googled for identity theft or sending you phishing emails. Your name can also be searched for public records or DNS whois information, which may disclose your home phone, address, date of birth, ID number and family members.

In summary, make it sure that somebody who finds your locked phone cannot discover who you are or interact as if they were you.

☑ Disable Medical ID: Health > Medical ID > Edit > Show When Locked

Some people think that letting anyone find out the owner of the phone is a good idea, since an honest person who finds your lost phone can easily contact you. However, you can always display a personalized message on your lock screen if you report your phone missing on iCloud.

☑ Enable "Find my phone": Settings > iCloud > Find my iPhone > Find My iPhone

Make sure that your phone will send its location just before it runs out of battery

☑ Enable "Find my phone": Settings > iCloud > Find my iPhone > Send Last Location

To finish this section, if you don't have the habit of manually locking your phone after you use it, or before placing it in your pocket, configure your iPhone to do it automatically:

☑ Enable phone locking: Settings > General > Auto-Lock

Harden the hardware

Your phone is now secure and won't sing like a canary when it gets into the wrong hands.

However, your SIM card may. SIMs can contain personal information, like names, phones or addresses, so they must be secured, too.

Enable the SIM lock so that, on boot, it will ask for a 4-digit code besides your phone password. It may sound annoying, but it isn't. It's just an extra step that you only need to perform once every many days, when your phone restarts.

Otherwise, a thief can stick the SIM in another phone and access that information and discover your phone number. With it, you may be googled, or they may attempt phishing attacks weeks later.

Beware that this strategy doesn't allow the phone to ping home after it has been shut down and turned on.

☑ Enable SIM PIN: Settings > Phone > SIM PIN

Enable iCloud. When your phone is associated with an iCloud account, it is impossible for another person to use it, dropping its resale value to almost zero. I've had some friends get their phones back after a casual thief tried to sell them unsuccessfully thanks to the iCloud lock and finally decided to do the good thing and return it.

☑ Enable iCloud: Settings > iCloud

If you have the means, try to upgrade to an iPhone 5S or higher. These phones contain a hardware element called Secure Enclave which encrypts your personal information in a way that can't even be cracked by the FBI. If your phone gets stolen by a professional, they won't be able to solder the flash memory into another device and recover your data.

☑ Upgrade to a phone with a Secure Enclave (iPhone 5S or higher)

Harden your online accounts

In reality, your online data is much more at risk than your physical phone. Botnets constantly try to find vulnerabilities in services and steal user passwords.

The first thing you must do right now is to install a password manager. Your iPhone has one built into the system, which is good enough to generate unique password and auto-fill them when needed.

If you don't like Apple's Keychain, I recommend LastPass and 1Password.

Why do you need a password manager? The main reason is to avoid having a single password for all services. The popular trick of having a weak password for most sites and another strong password for important sites is a dangerous idea.

Your goal is to have a different password for each site/service, so that if it gets attacked or you inadvertently leak it to a phishing attack, it is no big deal and doesn't affect all your accounts.

Just have a different one for each service and let the phone remember all of them. I don't know my passwords: Gmail, Facebook, Twitter, my browser remembers them for me.

☑ Use a password manager: Settings > iCloud > Keychain > iCloud Keychain

There is another system which complements passwords, called "Two-Factor Authentication", or 2FA. You have probably used it in online banking; they send you an SMS with a confirmation code that you have to enter somewhere.

If your password gets stolen, 2FA is a fantastic barrier against an attacker. Without your phone, they can't access your data, even if they have all your passwords.

☑ Use 2FA for your online accounts: manual for different sites

2FA makes it critical to disable SMS previews, because if a thief steals your phone and already has some of your passwords, he can use your locked phone to read 2FA SMS.

If you use iMessage heavily, this may be cumbersome, so decide for yourself.

☑ Disable SMS previews on locked phone: Settings > Notifications > Messages > Show Previews

Make it easy to recover your data

If the worst happens, and you lose your phone, get it stolen or drop it on the Venice canals, plan ahead so that the only loss is the money for a new phone. You don't want to lose your pictures, passwords, phone numbers, events...

Fortunately, iPhones have a phenomenal backup system which can store your phone data in the cloud or your Mac. I have a Mac, but I recommend the iCloud backup nonetheless.

Apple only offers 5 GB of storage in iCloud, which is poor, but fortunately, the pricing tiers are fair. For one or two bucks a month, depending on your usage, you can buy the cheapest and most important digital insurance to keep all your data and pictures safe.

iCloud backup can automatically set up a new phone and make it behave exactly like your old phone.

If you own a Mac, once you pay for iCloud storage, you can enable the "iCloud Photo Library" on Settings > iCloud > Photos > iCloud Photo Library for transparent syncing of all your pictures between your phone and your computer.

☑ Enable iCloud backup: Settings > iCloud > Backup > iCloud Backup

If you don't want the iCloud backup, at least add a free iCloud account or any other "sync" account like Google's, and use it to store your contacts, calendars, notes and Keychain.

☑ Enable iCloud: Settings > iCloud

Bonus: disable your phone when showing pictures

Afraid of handing your phone over to show somebody a picture? People have a tendency to swipe around to see other images, which may be a bad idea in some cases.

To save them from seeing things that can't be unseen, you can use a trick with the Guided Access feature to lock all input to the phone, yet still show whatever is on the screen.

☑ Use Guided Access to lock pictures on screen: Read this manual

This is not a thorough guide

As the title mentions, this is an essential blueprint for iPhone users who are not a serious target for digital theft. High-profile people need to take many more steps to secure their data. Still, they all implement these options too.

The usual scenario for a thief who steals your phone at a bar is as follows: they will turn it off or put it in airplane mode and try to unlock it. Once they see that it's locked with iCloud, they can either try to sell it for parts, return it or discard it.

Muggers don't want your data. However, it doesn't hurt to implement some security measures.

In worse scenarios, there are criminal companies specialized in buying stolen phones at a very low price and perform massive simple attacks to unsuspecting users to trick them into unlocking the phone or giving up personal data.

You don't need the same security as Obama or Snowden. Nonetheless, knowing how your phone leaks personal information and the possible attack vectors is important in defending yourself from prying eyes.

You have your whole life on your phone. In the case of an unfortunate theft, make it so the only loss is the cost of a new one.

Tags: security

Comments? Tweet  

Living in a disrupted economy

July 21, 2016 — Carlos Fenollosa

There is this continuing discussion on whether technology destroys more jobs than it creates. Every few years, yet another tech revolution occurs, journalists publish articles, pundits share their opinions, politicians try to catch up, and those affected always voice their concerns. These couple years have been no exception, thanks to Uber, Airbnb, and the called sharing economy.

I'm a technologist and a relatively young person, so I am naturally biased towards technological disruption. After all, it is people like me who are trying to make a living by taking over older jobs.

I suggest that you take a few minutes to read a fantastic article titled The $3500 shirt. That essay reveals how horrible some industries were before they could be automated or replaced by something better. Go on, please read it now, it will only take three minutes.

Now, imagine you had to spend a couple of weeks of your time to make a t-shirt from scratch. Would that be acceptable? I guess we all more or less agree that the textile revolution was a net gain for society. Nevertheless, when it occurred, some Luddites probably complained, arguing that the loom put seamstresses out of work.

History is packed with dead industries. We killed the ice business with the modern fridge. We burn less coal for energy, so miners go unemployed. And let's not forget the basis of modern civilization, the agricultural revolution, which is the only reason us humans can feed ourselves. Without greenhouses, nitrates, tractors, pest protection and advancements in farming, humanity would starve.

Admittedly, it transformed the first sector from a 65% in workforce quota into the current 10%. Isn't it great that most of us don't need to wake up before sunrise to water our crops? In hindsight, can you imagine proclaiming that the 1800s way of farming is better because it preserves farming jobs?

The bottom line is that all economic transformations are a net gain for society. They may not be flawless, but they have allowed us humans to live a better life.

So why do some characters fight against current industry disruptions if history will prove them wrong?


As a European and a social democrat, I believe that States must regulate some economies to avoid monopolies and abuses, supporting the greater good. Furthermore, I sympathize with the affected workforce, both personally and in a macroeconomic level. All taxi drivers suddenly going jobless because of Uber is detrimental to society.

However, it pains me to see that European politicians are taking the opposite stance, brandishing law and tradition as excuses to hinder progress.

Laws must serve people, not the other way around. If we analyze the taxi example, we learn that there is a regulation which requires taxi drivers to pay a huge sum of money up front to operate. Therefore, letting anybody get in that business for free is unfair and breaks the rules of the game. Unsurprisingly, this situation is unfair not because of the new players, but because that regulation is obsolete.

It isn't ethically right that somebody who spent a lot of money to get a license sees their job at risk. But the solution isn't to block other players, especially when it's regulation which is at fault. Let's sit down, think how to establish a transition period, and maybe even reimburse drivers part of that money with the earnings from increased taxes due to a higher employment and economic activity.

There is a middle ground solution: don't change the rules drastically, but don't use these them as an excuse to impede progress.

At the end of the day, some careers are condemned to extinction. That is a real social drama, however, what should we do? Artificially stop innovation to save jobs which are not efficient and, when automated or improved, they make the world better for everyone?


Us millennials have learned that the concept of a single, lifetime profession just does not exist anymore. Previous generations do not want to accept that reality. I understand that reconverting an older person to a new career may be difficult, but if the alternative is letting that person obstruct younger people's opportunities, that's not fair.

Most professions decline organically, by the very nature of society and economy. It is the politicians' responsibility to mediate when this process is accelerated by a new industry or technology. New or automated trades will take their place, usually providing a bigger collective benefit, like healthcare, education, or modern farming.

Our duty as a society is to make sure everyone lives a happy and comfortable life. Artificially blocking new technologies and economic models harms everyone. If it were for some Luddites, we'd be still paying $3500 for a shirt, and that seamstress would never have been a nurse or a scientist.

Tags: law, startups

Comments? Tweet  

The Elixir of concurrency

May 23, 2016 — Carlos Fenollosa

Elixir is a fairly young language that was born when José and a few Rails developers tried to create a modern language optimized for concurrent, distributed, lightweight processes

They wanted a modern Ruby-like syntax with a well-tested process manager, the Erlang VM. The result is Elixir, defined as a dynamic, functional language designed for building scalable and maintainable applications, a correct but vague affirmation which doesn't do justice to its power and elegancy.

I recently compared the move to Elixir from Python as a similar leapfrog to moving to Python from Java. It feels like something new, modern, powerful, with killer features that you don't want to renounce to.

In Python I found a REPL, list comprehensions, a super clean syntax and decorators. Elixir brings lightweight supervised processes, pattern matching, a fully functional programming language, pipes and a terrific build tool: mix

If you've never written functional code, the jump is significant. I took a Scala course a couple years ago and I've needed almost two full weeks to write production code in Elixir. The language is young, Stack Overflow is of no help —no kidding, that is a big deal—, and there are few libraries in Github.

A small community also comes with some upsides: people are more motivated and willing to help, centralized tools like forums and IRC channels are still manageable, and you may even suggest changes to the language for upcoming versions.

What is Elixir for?

I had a middle school teacher who said that you can't define something by stating what is't not. However, in programming, mentioning use cases which are not suitable for the language is a good way to start.

Elixir is probably not the first choice for single core software: math calculus, CPU-intensive apps or desktop applications. Since it's very high level, systems programming is also out of the picture.

Elixir is great for web applications, standalone or using the Phoenix framework —Elixir's Rails—. It really shines for building highly scalable, fault-tolerant network applications, like chats, telecommunications or generic web services.

Why is that? Thanks to the Erlang VM, processes are really tiny, each one is garbage collected with a low latency, they communicate by sending location-independent messages over the network using the VMs (you can run result = Machine2.Module.function(params) on Machine1), and spawning and managing these processes is effortless thanks to some of its abstractions.

Finally, Elixir's basic modules also shine: Plug and Router for managing HTTP requests, Ecto for relational databases and ETS and Mnesia for distributed in-memory databases.

Many recommend Elixir if only for Phoenix, but I found that for most backend applications it is enough to use Plug and Router. Phoenix is impressive but I believe it's a mistake to jump right into it without trying the base modules first, so my recommendation for beginners is to hold on Phoenix until you really need it.

Elixir's novelty, the pipe operator, is a fantastic approach to working with state in a functional manner. Instead of running readlines(fopen(user_input(), "r")).uppercase().split(), try the more readable user_input |> fopen("r") |> readlines |> uppercase |> split.

It is a language which was clearly designed to stand on the shoulders of giants, while providing modern capabilities for developers.

Elixir's abstractions

To store centralized <key, value>-like data, instead of a Singleton, Elixir's provides an Agent. It keeps state in memory and many processes can access and modify it without concurrency issues.

The language can spawn processes much like threads, using spawn_link, but you probably don't want to do that. You'd rather use a Task, which is basically async/await, or a Gen(eric)Server, a very cool abstraction that receives requests from other processes, spawns helper mini-servers and processes the results in parallel, for free.

All tasks can be controlled using the Supervisor, which holds other abstractions as its "children" and automatically restarts them when they crash.

Finally, your code is contained inside a single project which can manage different apps, with modules that hold functions. No packages, no classes, no objects. Modules, functions, structs and basic data types.

Dependency management is straightforward thanks to mix; builds and testing are handled by mix too. As opposed to other multi-tools like gradle, this one is really fast.

Is that too much to process? I felt that at first, too. Give it some time and your brain will eventually think in terms of Supervisors which manage GenServers which spawn Agents and Tasks when needed.

Let it crash

Elixir's mantra is to let processes crash. I found it shocking and counter-intuitive, but with some explanation it makes a lot of sense.

Neither developers want their code to crash nor Elixir promotes writing bad code. However, let's agree that there are many reasons besides bad programming which can make a software crash. If we have a server which runs stuff and at some point we have, say, 100 connections every second, one might crash eventually because of a bug in any component, hardware issues, a cosmic ray, or Murphy's law.

The question is: in the event of an unfortunate, unavoidable crash, how will your system react?

  1. Bring everything down?
  2. Try to capture the error and recover?
  3. Kill the crashed process and launch another one in its place?

For example, C uses approach 1. Most modern languages with Exceptions like Java and Python use 2. Elixir uses 3. This is not suitable for all environments, but it is perfect for those use cases which fit Elixir: concurrent network processes.

With Elixir, a single failure never brings the system down. What's more, it automatically restarts the crashed process, so the client can instantly retry and, unless there is a reproducible bug in your code, the fresh process will finish without an issue.

The bottom line is: a single client may be unlucky and crash at some point, but the rest of the system will never notice.

How to start?

Let's get our hands dirty. After reading many sites, watching hours of video and following a dozen tutorials, here are the resources I found the most valuable. I'd suggest following this order.

Getting started

  1. Madrid Elixir Meetup 2016-03. If you understand Spanish, this is the best intro to Elixir. Otherwise, watch All aboard the Elixir Express! which is a bit outdated but very comprehensive.
  2. Official "Getting Started" guide. It's the best and the most current. Follow it from start to finish, including the advanced chapters.
  3. Elixir School. A nice complement to the official guide. Most things are very similar, but the different approach on OTP will help you understand it better.
  4. Understanding Elixir's GenServer and Elixir's supervisors, a conceptual understanding are two short reads with yet another explanation of OTP features.
  5. Elixir Cheat Sheet. The best one out there

First projects

  1. vim-elixir-ide. Elixir support for vim, not the best plugin but suitable for beginners.
  2. Elixir examples. The Elixir guide covers all these, but it's handy to have common idioms on a single page: "string to list", "concatenate list", "optional function parameters", etc.
  3. Portal Game by José Valim. A complement to the sample project on the official guide.
  4. Elixir Koans and Exercism are mini exercises that you can use to improve your Elixir agility. On the same line, Elixir Golf proposes weekly puzzles to solve.
  5. Learning Elixir. Joseph Kain has a ton of content with mini projects and examples you can follow. Top quality.
  6. Excasts and Elixir sips have short screencasts that you can check out for reference
  7. ElixirConf videos contain very interesting talks which may be overwhelming for beginners, but are worth a look later on.
  8. Install Elixir and Phoenix on OSX. If you want to use Phoenix on OSX, you may need this help
  9. Phoenix Official Guide. Phoenix isn't necessary for simple web services, you can use Plug. But for large projects you'll need a framework. Nothing like the official guide.

Getting help

  1. Awesome Elixir. A list of Elixir resources, where I found many of these.
  2. Elixir Tip and Elixir Status regularly link to Elixir-related articles and videos, and Plataformatec Elixir posts is where the language authors share news and tips.
  3. If you have questions about code, try the Elixir forum first, the IRC channel or Slack. The developers would like to transition all help requests out of the Mailing list, which you can use for language-related discussions.
  4. /r/elixir if you're into Reddit

Closing thoughts

I think that's all for the moment. I hope this post can help some beginners to get their hands on the language and start writing production code as soon as possible.

For anyone who wants to know what's all the Elixir fuss about, it's difficult to explain, especially for somebody like me who has been programming in imperative languages all his life.

When I recommended Elixir to a friend, he replied, "A highly concurrent, functional language using the Erlang VM? Don't you have something more exotic?". That's right. Elixir is exotic and use-case specific.

Unlike Python, which is my favorite imperative language and ecosystem, I can't recommend Elixir for everyone. Not everybody can spare a couple weeks to get started. Many libraries for common use cases are missing: there is nothing equivalent to Numpy or Matplotlib, and modern applications are built on top of dozens of libs, not everyone has the time or will to write library code. Fortunately, at Paradoxa I am my own boss and I make the tech decisions :)

For hackers or tinkerers it's definitely worth a look, it "won't change your perspective" like Lisp, but it will make you see that writing concurrent code doesn't need to be difficult, and that better tooling is definitely possible.

I bet Elixir will be the foundation of most devops stacks in a few years, when developers realize that the future's bottleneck won't be the CPU, but rather the number of concurrent processes and connections your backend can manage. With Elixir you only need to boot another machine in your network and let the exotic Erlang VM handle the rest.

Tags: programming, learning

Comments? Tweet