Last conversion of the Campfire from my token stash goes to JavaScript/Express. As with the others, I'm sure you can find more to optimize. But this gives you a sense of what order of magnitude these options exist within with agent-led development. https://t.co/UUapL99SJE https://t.co/2H2NXfeZog

— DHH (@dhh) October 5, 2026

I won’t defend him this time. A one-shot conversion, done by an agent, with no fine-tuning at all, doesn’t measure language: it measures how much the model knows each stack’s idioms. It’s rage bait and he knows it. That said, I do the same from time to time, and so do half his critics, so we’re all hypocrites and life goes on.

And the weak spot of his methodology became a topic of its own: several people noticed the Rust version had gone through far more agent iterations than the others, meaning more optimization rounds. @kurtome answered in practice: he took the Elixir version and sped up the main page requests 10x, with obvious optimizations that no first conversion does, like caching and async database connection pooling.

@dhh Here you go https://t.co/gUtK7Z7zG7

got a 10x speed up on the Room page requests, and sped up all the other requests significantly as well. the biggest gains came from some no brainers: implement caching and pool the db connections with proper async https://t.co/mbp94q9N0s

— kurt (.com) (@kurtome) October 5, 2026

His comparison, then, wasn’t “Rust versus Elixir”: it was “Rust after N tuning rounds versus Elixir after one.” And that lays bare the truth the thread pretends to ignore: no real software ships from half a dozen prompts. Coding with AI shines when an experienced engineer steers the agent toward their goal, and that means dozens of iterations, not three.

The good part is that the bait was useful to me. I’ve been doing LLM benchmarks and decision-model benchmarks, but I had never measured, in a reproducible way, the question that thread pretended to answer: does it matter which language you pick when the one writing the code is an LLM? My short answer was already out in two posts of mine:

Pressure under load is not a trivial problem to solve from scratch and I recommend you do not attempt to. It's a solved problem already and all solutions converge to what Elixir already brings out of the box. So yes. I use Rust all the time, but if I had to solve high scalability problems (particularly high volume message queues), I would think of Elixir first.

— Fabio Akita (@AkitaOnRails) October 9, 2026

Language wars are tiresome. And the reality is simpler if you're doing AI assisted coding.

- system level (drivers, fs, daemons, etc): Rust. C++ is you're extending existing legacy
- windows only desktop apps: C#
- iOS/MacOS/iPadOS: Swift
- multi-OS: Rust/Tauri, Qt/C++, Flutter, Electron (pick your poison)
- simple websites: Node/Typescript/Bun/Deno
- highly scalable backend: Elixir, parts in Rust, parts in Go
- CLI tools: Rust

Everything else, by default, Rust. Unless it's an edge case.

— Fabio Akita (@AkitaOnRails) October 9, 2026

There’s no better language, there’s the right tool

That list from the post isn’t a ranking, it’s a terrain map. Every language and every framework was designed for a use case, and none covers them all. That was true when you wrote every line by hand and it stays true with agents. The LLM doesn’t change the physics of the problem: it only changes who’s typing.

And the map goes beyond language. A real system has infrastructure around it: Kubernetes when scale asks for it, SQLite by default because most apps never need more than that, Postgres with pgBouncer when they do, Redis for cache and key-value, RabbitMQ or NATS for queues. Picking “the stack” on day zero and expecting it to last ten years is fantasy. It doesn’t, and that’s fine.

Personal example: new web app, I open Ruby on Rails without thinking twice. It’s the most complete end-to-end solution there is for the web, and with an LLM writing most of the code, the framework’s productivity weighs even more than the language’s. As the system grows, I swap the parts that ask for another tool: a high-volume queue leaves Rails’ side and goes to something from Erlang land, starting with RabbitMQ, which already solves the hard backpressure part for me.

No good software is finished: it grows. Usage dictates what you replace, rewrite and add. The final architecture is never the first commit’s, and whoever tries to get everything right in the first commit only produces dead documents.

The token superstition

DHH’s thread resurrected two beliefs that have been circulating since LLMs became work tools:

• “Script languages are better for AI because the lean syntax spends fewer tokens.”
• “Strongly typed languages are better for AI because the compiler catches the errors, so you need less testing and less repair.”

Both sound reasonable and both are superstition until someone measures. I was betting against both, from experience: a good compiler helps, but it doesn’t do magic, and syntax conciseness was never the bottleneck of an agent that reads and writes files all day.

What really separates a project that moves from a project that spins in circles is test discipline. I’ve written that TDD became a technical obligation, not a philosophy, and this post’s benchmark is one more data point in the same direction. It doesn’t matter whether you do test-first or test-after: what matters is that the suite grows with the code, that every user-reported bug becomes a regression test before the fix, and that you never make the same mistake twice. Software without users is dead software, so release early, release often, break fast and fix even faster. AI closes exactly that gap: the reproduce-test-fix cycle got cheap. QA became the programmer’s most valuable skill, far above “knowing Rust syntax.”

Discipline beats compiler and beats language. That was my hypothesis. Time to measure.

The benchmark

I built a reproducible experiment, public in the vibe-coding-benchmarks repository, with per-attempt results published in JSON and the raw traces kept out of the repo for credential containment. It’s not a language championship, and the design itself says so: it’s a snapshot of one real task, measuring what can be honestly measured.

The task

One task only, on purpose: maintain a multi-tenant ledger engine. The agent receives working v1 code and has to take it to a v2 contract, with per-tenant identities, authorization, serializable snapshots, a unique secondary index with alias swap, a capacity-bounded journal, torn-group recovery, checkpointing, legacy schema migration and durable retry IDs. It’s not FizzBuzz: it’s the kind of system that demands invariants across operations, exactly where LLMs tend to trip.

Correctness is judged by 43 test stories applied after collection, out of the agent’s sight: the 27 public ones from T1 plus 16 hidden ones. Passing is 43 out of 43. The conditions:

• T0: prose contract, input and output examples, a working launcher. No test suite.
• T1: the same, plus 27 public stories and a driver to run them.

Each attempt had a 15-minute budget, with at most two 3-minute continuations. In practice, none of the 112 needed a continuation.

The languages and the models

Seven languages in three strata, with pinned toolchains and Rust in safe mode:

• Typed natives: Rust (safe, pinned), Haskell (pinned GHC).
• Typed VMs: C# (with nullable analysis), Kotlin, TypeScript (strict, noUncheckedIndexedAccess, exactOptionalPropertyTypes).
• Dynamic scripts: Python and Ruby, no checker at all.

Four model configurations, all on subscriptions, no paid fallback: Kimi K3 (high), GLM 5.3 via OpenCode (high), Claude Opus 5.5 (medium) and Codex with gpt-6 Sol (medium). Crossing everything: 7 languages × 4 models × 2 conditions × 2 repetitions = 112 attempts.

What this design doesn’t measure

Before the numbers, the brakes, because they’re part of the result. It’s a single task, with two repetitions per cell: a small sample by construction. Token coverage is partial in some models (missing data is not zero). The harness changed mid-campaign across three instrumentation epochs. And provider delays contaminate wall time: one of the Opus attempts spent half the clock waiting for quota release. All numbers below are descriptive, and I point out where they can’t support a conclusion.

There’s one additional brake, on purpose: the iteration level. My design measures what can be reached in all languages with similar iteration levels, on a short budget, with no human tuning afterwards. It’s a photograph of the floor, not the ceiling. Any of the 112 implementations probably improves with more agent rounds, and the four queue stacks likewise: what I publish is what each language delivers “out of the factory” under the same effort, not the maximum a patient engineer would extract from it.

Mandatory disclaimer before quoting any number from this post around: every result that follows holds for this benchmark, with this task, these models, these prompts and this hardware. Your case may vary violently. If your task doesn’t look like a ledger engine, if your model isn’t one of these four, if your queue isn’t my queue, the numbers don’t transfer. The code, the tasks, the grades and the step-by-step to reproduce are all in the GitHub repository. And if you disagree with some result and that disagreement matters to your company or your project, the only honest attitude is the one I had: stop arguing and run your own benchmark.

Result 1: on correctness, the frontier models tied at the ceiling

Reading: Sol passed all 28 eligible attempts and Opus its 25 (the 3 failures were infrastructure, not code). 53 of 53 eligible attempts correct between the two. The task, made to be hard, was too easy to separate the frontier models, let alone the languages: with everyone passing everything, there’s no correctness ranking to publish.

The difference showed up in the weaker models, and in the direction my hypothesis predicted: provided tests helped. Kimi and GLM, with the public suite in hand (T1), delivered 24 correct artifacts against 16 without it (T0). Kimi, in particular, passed all 14 attempts with provided tests. It’s descriptive, not causal, but it’s the cleanest signal in the whole benchmark: what changed the outcome wasn’t the language, it was the presence of tests.

Per language, within each model, correctness scattered between 1/4 and 4/4 with no pattern that repeats across models. The three semantic failures I audited in depth are instructive: a GLM C# and a Kimi Python lost the same story by violating the same cross-operation invariant, and a Kimi Haskell missed a checkpoint path. None of the failures was a type error.

Where the model erred, it erred on domain invariants: the kind of error no compiler catches, in typed or dynamic languages. The myth that strong typing spares you the work of understanding the problem dies right there.

Result 2: concise language saved no tokens

Here’s the table that kills the first superstition. Medians per language on Sol, four valid attempts per cell:

Ruby, the most concise in lines (a bit over half of Rust’s), cost practically the same tokens as Rust on Sol, and on Opus it was the most expensive of all, about 205 thousand median tokens against C#’s 94 thousand. The cheapest were TypeScript and Python on Sol, C#, Haskell and Kotlin on Opus. If “fewer source code lines = fewer LLM tokens” were a law, Ruby should have led by a wide margin. It finished last.

And the second superstition? I audited the traces looking for the compiler-guided repair cycles that strong typing supposedly avoids. Result: zero compilation-error repair cycles in the audited Opus attempts, across Ruby, Rust and Haskell. Rust needed no compiler-driven fix at all; Haskell had one warning cleanup; Ruby spent its extra tokens on voluntary performance tuning. The frontier model almost always writes code that compiles on the first try, and the real effort goes to verification, not to appeasing the compiler.

One counterpoint that closes the coffin on the “cheap is good” thesis: GLM’s lowest token-consumption cells were the failures. TypeScript T0 at 50 thousand tokens looks efficient until you see it’s two unmodified starters because the model blew the output limit. Low tokens can mean early surrender.

Result 3: under load, everyone tied at the queue

Now for DHH’s question: Rust “beats” Elixir and Ruby when the subject is throughput. I built the separate track for that: four queue workers against a real RabbitMQ, written with agent help, each in its natural idiom:

• Elixir with OTP and Broadway
• Rust with Tokio and Lapin
• Rails with Bunny
• Plain Ruby with Bunny (control)

A generator published 10 thousand 4 KB messages to RabbitMQ, at 200 per second, and each worker consumed as fast as it could, hitting the effects on an HTTP/SQLite collector with a nominal 50 ms delay, limited to 2 CPUs and 1 GB of RAM. Two concurrency profiles:

On the 8-handler profile, the four stacks delivered the same throughput, 93 to 95 effects per second, and drained the whole queue. The apparent bottleneck was the shared collector (a Python lock, a single SQLite writer), not the languages. On the 64-handler profile, all of them left a backlog and blew the client timeout en masse, so I marked those cells as not qualified: the strong protocol, with a group-commit collector and a 50 to 800 msg/s ramp, is designed but not executed.

The only difference left standing in the numbers was memory: Rust held the peak at 16 MiB where the BEAM spent 135. It’s a real Rust advantage in density, with the caveat that a cgroup peak is not a language footprint. On throughput, which was the thread’s claim, a technical tie.

And the tie deserves an honest reading: it may only mean my pressure was too little. With the shared collector holding everything around 95 effects per second, any stack capable of more than that shows up as “equal.” What this test shows is where the bottleneck was, not where each stack breaks. To find each one’s ceiling I’d need the strong protocol, with a group-commit collector and a load ramp, which is designed but not executed. Until then, “all stacks perform the same” is the wrong conclusion; “all of them stayed above what my test could measure” is the right one.

And the question that interested me most, “does the agent work less to deliver a reliable queue consumer in Elixir than in Rust?”, was deliberately left unanswered: these four implementations were written to validate the environment, not as a controlled coding attempt. Comparing AI effort between Elixir and Rust would require the same ledger matrix running on the queue task. It’s on the list.

So, does the language matter?

With the data on the table, the thread’s three questions can be answered:

Is there a better language for programming with LLMs? None showed up. In the frontier models, correctness and effort varied less between languages than between models. Where there was a correctness difference, it followed the presence of provided tests, not syntax. “No general language ranking is established” is the report’s official sentence, and it’s the correct one.

Do the token-economy theses hold? Neither one. Conciseness didn’t save: Ruby cost as much as Rust, or more. Strong typing spared no repair: there was no type repair to spare. The models’ effort went to verification and understanding the contract, and that has no language.

Was DHH right? In what he measured, he measured what an agent delivers for free on each stack, without tuning, and called that a verdict on language. In my test, with real load and honest measurement, the four stacks tied on throughput in the stable regime, and only memory differentiated Rust. If his thread proves anything, it proves that agent conversion measures the model, not the language.

My bets from the introduction came out standing: testing and discipline are the lever, the compiler is a supporting actor, and the right tool depends on the problem. New web app is still Rails in my hand, heavy queue is still Erlang territory, low-level system is still Rust, and the LLM writes any of them with the same competence. Whoever spent the week fighting for their favorite language on X fought over nothing.

And my daily practice tells the same story. I have a personal bias toward Rust and even so I’ve published more than 30 open source projects in all kinds of stacks: ai-memory is Rust, FrankMD is Ruby on Rails, ai-memory-web and ai-jail-web are Node.js. In none of them did I feel any relevant difference that came from the stack: the wins were similar, the errors too, and the errors were always understandable. Every error became an invariant, a rule, a skill improvement. It never became a language switch.

So forget “token savings” and “typing superstition.” Do real software engineering: pick the right tool for your specific problem, apply proper engineering technique, and any reasonable agent will perform more or less the same. And stop wasting time arguing over language-war rage bait on the internet. Be a real software engineer and do real engineering. That’s it.

The per-attempt data, the full methodology and the evidence limits are in the repository. If you run the matrix with another task and find a ranking I didn’t, show me.

José Valim kindly stepped in and posted a valuable comment, which I will paste here:

Have you tried reducing the @max_demand instead? @max_demand is how much data you exchange between stages. If you set it to 60, it means you are sending 60 items to one stage, 60 items to the other and so on. That gives you poor balancing for small collections as there is a chance all items end-up in the same stage. You actually want to reduce max_demand to 1 or 2 so each stage gets small batches and request more than needed. Another parameter you usually tune is the stages: ... option, you should set that to the amount of connections you had in poolboy in the past.

However, I believe you don’t need to use Flow at all. Elixir v1.4 has something called Task.async_stream which is a mixture of poolboy + task async, which is a definitely better replacement to what you had. We have added it to Elixir after implementing Flow as we realized you can get a lot of mileage out of Task.async_stream without needing to jump to a full solution like Flow. If using Task.async_stream, the max_concurrency option controls your pool size.

And, obviously, he is right. I misunderstood how Flow works. It’s meant to be used when you have a bazillion items to process in parallel. Particularly processing units that can have high variance and, hence, a lot of back-pressure not only because there is a lot of items to process, but because their running times can vary dramatically. So, it’s one of those cases of bringing a cannon to kill a fly.

What I wasn’t aware is the existence of Task.async_stream and its companion Task.Supervisor.async_stream if I need to add more control.

Let’s backtrack a bit.

Dealing with collections in Elixir

Erlang is a beast. It provides all the building blocks of a real-time, highly-concurrent, operating system! Really, what it provides out of the box is way more than any other language/platform/virtual machine, ever. You don’t get that much for free on Java, or .NET or anything. You have to assemble the pieces manually, spend hundreds of hours tweaking, and still pray a lot to have everything working seamlessly.

So, you have distributed systems to build? There is no other option, really. Do Erlang, or suffer in hell.

Then, Elixir steps this up a notch creating a very reasonable and simple to use standard library that makes the coding part actually enjoyable. This is a killer combo. You need to do the next Whatsapp? You need to do the next Waze? You need to rebuild Cassandra from scratch? You need to create stuff like Apache Spark? Do Elixir.

In Erlang, you need to solve everything using GenServer. It’s a neat abstraction from OTP. You are required to understand OTP intimately. There is no shortcut here. There is no Erlang without OTP.

That said, you can start simple and scale without so much hassle.

Usually, everything starts with Collections, or more correctly, some kind of Enumeration.

Just like my simple Workflow.pages/1 function which iterates through a list of chapter links, fetch each link, parse the returning HTML and extracts the collection of page links within that chapter, reducing the sub-lists into a full list of page links.

If I know the collection is small (less than 100 items, for example) I would just do this:

def pages({chapter_list, source}) do
   pages_list = chapter_list
     |> Enum.map(&Worker.chapter_page([&1, source]))
     |> Enum.reduce([], fn {:ok, list}, acc -> acc ++ list end)
   {pages_list, source}
end

And that’s it. This is linear. It will sequentially process just one link at a time. The more chapter links, the longer it will take. Usually I want to process this in parallel. But I can’t fire a parallel process for each chapter link, because if I receive 1,000 chapter links and fire them all, it will be a Denial of Service and I will certainly receive hundreds of time outs.

You can run into 2 main problems when you need to iterate through a big collection.

• If your collection is humongous (imagine a gigabyte long text file that you need to iterate line by line). For that you use Stream instead of Enum. All functions look almost exactly the same, but you will not have to load the entire collection into memory and you will not keep duplicating it.

• If your processing unit takes a long time. Now that you solved not blowing off your memory usage, what if you have slow jobs while iterating through each item in the collection? That’s our case, where the collection is rather small, but the processing time is long as it’s fetching from an external source on the internet. It can take milisseconds, it can take a few seconds.

One way to control this is through the use of “batches”, something along these lines:

def pages({chapter_list, source}) do
  pages_list = chapter_list
	|> Enum.chunk(60)
	|> Enum.map(&pages_download(&1, source))
	|> Enum.concat
  {pages_list, source}
end

def pages_download(chapter_list, source)
   results = chapter_list
     |> Enum.map(&Task.async(fn -> Worker.chapter_page([&1, source]) end))
     |> Enum.map(&Task.await/1)
     |> Enum.reduce([], fn {:ok, list}, acc -> acc ++ list end)
   {pages_list, source}, results	
end

This is just for the example, I have not compiled this snippet to see if it works, but you get the idea of chunking the big list and processing each chunk through Task.async and Task.await.

Again, for small lists, this should be ok (thousands) and where each item does not take too much to process.

Now, this is not very good. Because each chunk must finish before the next chunk begins. Which is why the ideal solution is to keep a constant amount of jobs running at any given time. To that end, we need a Pool, which is what I explained in Part 2: Poolboy to the rescue!.

But implementing the proper way to keep the pool entirely filled requires some boring juggling between Poolboy transactions and Task.Supervisor.async. Which is why I was interested in the new Flow usage.

The code does come clean, but as I explained before, this is not the proper use case for Flow. It’s better when you have to iterate over tens of thousands of items or even an infinite stream (you have incoming traffic of requests in need of parallel processing, for example).

So, finally, there is a compromise. The solution between the simple Task.async and Flow is Task.async_stream which works like a pool implementation, where it keeps a max_concurrency of jobs running in a stream. The final code becomes way more elegant like this:

def pages({chapter_list, source}) do
  pages_list = chapter_list
    |> Task.async_stream(MangaWrapper, :chapter_page, [source], max_concurrency: @max_demand)
    |> Enum.to_list()
    |> Enum.reduce([], fn {:ok, {:ok, list}}, acc -> acc ++ list end)
  {pages_list, source}
end

And this is the final commit with the aforementioned changes.

Conclusion

The implementation with Task.async_stream is super simple and the times finally became the same as before.

84,16s user 20,80s system 138% cpu 1:15,94 total

Way better than the more than 3 minutes it was taking with Flow. And this is not because Flow is slow, it’s because I was not using it correctly, probably shooting a big chunk into a single GenStage and creating a bottleneck. Again, only use Flow if you have enough items to put hundreds of them into several parallel GenStages. We are talking about collections with tens of thousands of items, not my meager pages list.

There is a small tweak though. To fetch all chapter and page links I am using a max_concurrency of 100. Timeout is the default at 5000 (5 seconds). That works because the returning HTML is not so big and we can parallelize that much on a high bandwidth connection.

But the images downloading procedure at Workflow.process_downloads I had to cut max_concurrency in half and increase the timeout up to 30 seconds to make sure it wouldn’t crash.

Because this is a simple implementation there is no crash recovery and no retry routine. I would have to replace this implementation with Task.Supervisor.async_stream to regain some control. My original implementation was more cumbersome but I had places to add retry mechanisms. So again, it’s a compromise between ease of use and control, always. The more control you have, the worse the code becomes.

This is a simple example exercise, so I will keep it at that for the time being.

As a quick recap, the exercise is that I want to scrape MangaReader.net for a bunch of images, organized in pages and chapters, and in the end it should compile organized PDFs so I can load them on a Kindle device.

Web scraping is a simple loop of HTTP GETs over a ton of URLs, scraping the HTML, and fetching more URLs to download.

In many simple languages, people usually solve this naively in 2 ways:

• A simple nested loop. One single thread, sequential fetches. So if you have 5,000 links and each links takes 10 seconds to fetch, it’s basically 10 * 5,000 = 50,000 seconds, which is a stupid long time.
• A simple spawn of processes, fibers, threads or parallel I/O in a reactor, all at once. An attempt to parallelize all fetches at once.

Everybody probably agrees that the first option is stupid. Now, the second one is tricky.

The tricky part is CONTROL.

Anyone in Go would say “oh, this is easy, just put a loop and spawn a bunch of goroutines” or anyone in Node.js would say “oh, this is easy, just put a loop, make the fetch - they all will run asynchronously - and add callbacks, a basic async/await.”

They’re not wrong, but this is still too naive an implementation. It’s trivial to trigger hundreds or thousands of parallel requests. Now, what happens if one fails and you have to retry? What happens if the MangaReader has a throttling system that will either start cutting down connections or timing them out? Or if your internet bandwidth is just not enough, and after a certain amount of requests you start having diminishing returns and time outs?

The message is: it’s damn trivial to spawn parallel stuff. it’s damn complicated to control parallel stuff.

This is why, in my first implementation in Elixir, I introduced a complicated implementation using a combination of a custom GenServer, Elixir’s own Task infrastructure for async/await pattern, and Poolboy to control the rate of the parallelism. This is how you control the bottleneck out to slow resources: using pools and queues (which is why every good database has a connection pool, remember C3P0?)

This is one snippet of my older implementation:

def chapter_page([chapter_link, source]) do
  Task.Supervisor.async(Fetcher.TaskSupervisor, fn ->
    :poolboy.transaction :worker_pool, fn(server) ->
      GenServer.call(server, {:chapter_page, chapter_link, source}, @genserver_call_timeout)
    end, @task_async_timeout
  end)
end

Yes, it’s very ugly, and there are boilerplates for the GenServer, the custom Supervisor to initialize Poolboy and so on. And the higher level workflow code looks like this:

def pages({chapter_list, source}) do
   pages_list = chapter_list
     |> Enum.map(&Worker.chapter_page([&1, source]))
     |> Enum.map(&Task.await(&1, @await_timeout_ms))
     |> Enum.reduce([], fn {:ok, list}, acc -> acc ++ list end)
   {pages_list, source}
end

So, inside the Worker module each public method wraps the GenServer internal calls into a Task async and in the collection iteration we add Task.await to actually wait for all parallel calls to finish, so we can finally reduce the results.

Elixir now comes with this very interesting GenStage infrastructure that promises to replace GenEvent and the use case is when you have a producer-consumer situation with back-pressure. Basically when you have slow endpoints and you would end up having to control bottlenecks.

Then, Flow is an easier high abstraction that you can use almost the same way you would use Enum in your collections, but instead of sequential mapping, it takes care of parallel traversing and control of batches. So the code is very similar but without you having to control the parallelization controls manually.

This is the full commit where I could remove Poolboy, remove my custom GenServer, reimplement the Worker as simple module of functions and then the workflow could get rid off the async/await pattern and use Flow instead:

def pages({chapter_list, source}) do
   pages_list = chapter_list
     |> Flow.from_enumerable(max_demand: @max_demand)
     |> Flow.map(&MangaWrapper.chapter_page([&1, source]))
     |> Flow.partition()
     |> Flow.reduce(fn -> [] end, fn {:ok, list}, acc -> acc ++ list end)
     |> Enum.to_list()
   {pages_list, source}
end

The only boilerplate left is the Flow.from_enumerable() and Flow.partition() wrapping the Flow.map, and that’s it!

Notice I configured @max_demand to be 60. You must tweak it to be larger or smaller depending on your internet connection, you have to experiment it. By default, Flow will trigger 500 processes in parallel, which is way too much for web scraping on a normal home internet connection and you will suffer diminishing returns. That’s what I had to do previously with Poolboy, by initiating a pool of around 60 transactions at most.

Unfortunately not everything is as straight forward as it seems. Running this new version on the test mode I get this result:

58,85s user 13,93s system 37% cpu 3:13,78 total

So a total time of more than 3 minutes, using around 37% of the available CPU.

My immediate previous version using all the shenanigans of Poolboy, Task.Supervisor, GenServer, etc still gives me this:

100,67s user 20,83s system 152% cpu 1:19,92 total

Less than HALF the time, albeit using all my CPU cores. So my custom implementation still uses my resources to the maximum. There is still something in the Flow implementation I didn’t quite get right. I already tried to bump up the max_demand from 60 up to 100 but that didn’t improve anything. Leaving it to the default 500 slows everything down to more than 7 minutes.

All in all, at least it makes the implementation far easier on the eyes (hence, way easier to maintain), but either the Flow implementation has bottlenecks or I am using it wrong at this point. If you know what it is, let me know in the comments section below.

I’ve published an article about ExAdmin and Coherence and another on Deploying Elixir to DigitalOcean this month.

The idea is very simple, it is a homage to Pusher. If you used Pusher before, this is very similar (albeit way less feature complete, of course).

I built an entire solution inspired by Pusher, using the Phoenix framework, deployed to Digital Ocean and you can test it out right now, just sign up at expusherlite.cm42.io.

Once you sign up, you will have a secret token (don’t disclose that, of course) and you will be under an Organization. Then you can go on and create Applications within that Organization. Each Application will have a unique token to identify it.

Now, let’s say you want to create a Rails application with a Chat feature. Any version of Rails, you don’t need 5.0 and you don’t need ActionCable.

First off, let’s configure config/secrets.yml:

development:
  secret_key_base: b9a1...e7aa
  pusher_host: "expusherlite.cm42.io"
  org_id: acme-inc
  app_key: 0221...f193
  secret_token: 4036...f193
...
production:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
  pusher_host: <%= ENV['PUSHER_LITE_HOST'] %>
  org_id: <%= ENV['PUSHER_LITE_ORGANIZATION'] %>
  app_key: <%= ENV["PUSHER_LITE_APP_KEY"] %>
  secret_token: <%= ENV["PUSHER_LITE_SECRET_TOKEN"] %>

Replace the pusher_host, org_id, app_key, and secret_token for the ones you created before.

Now I want to add a PageController:

require "net/http"
require "uri"
class PageController < ApplicationController
  def index
    uri = URI.parse("http://#{Rails.application.secrets.pusher_host}/api/sessions")
    response = Net::HTTP.post_form(uri, {"token" => Rails.application.secrets.secret_token})
    @guardian_token = JSON.parse(response.body)["jwt"]
    Rails.logger.info @guardian_token
  end
end

(If you’re connecting to my online server, you must use SSL, so change the URL above for “https”)

What this piece does is submit the secret token in the server-side, to my service, to get a JSON Web Token (JWT) back. Now you can pass this JWT to the front-end to enable authentication.

In the front-end we can have this simple app/views/page/index.html.erb:

<h1>Ex Pusher Lite - Rails Integration Example</h1>

<script type="text/javascript" charset="utf-8">
  window.guardian_token = "<%= @guardian_token %>";
  window.org_id = "<%= Rails.application.secrets.org_id %>"
  window.app_key = "<%= Rails.application.secrets.app_key %>";
  window.pusher_host = "<%= Rails.application.secrets.pusher_host %>";
</script>

<div id="chat" class="fixedContainer">
</div>

<input type="text" name="name" id="name" value="" placeholder="Name"/>
<input type="text" name="message" id="message" value="" placeholder="Message"/>
<input type="checkbox" name="channel" id="channel" value="api"/>
<label for="channel">send through API</label>

Super simple, we can tweak the CSS (app/assets/stylesheets/application.css) just to make it look nicer:

...
.fixedContainer {
  height: 250px;
  width: 100%;
  padding:3px;
  border: 1px solid black;
  margin: 5px;
  overflow: auto;
}

body {
  font-family: Helvetica, Arial
}

Finally, we need to load the main Javascript from the ExPusherLite server, so edit the layout at app/views/layouts/application.html.erb and add this line right after the closing </body> tag:

<script src="http://<%= Rails.application.secrets.pusher_host %>/js/pusher.js"></script>

And we can now use this Javascript in the app/assets/javascripts/application.js to hook everything up. This is the relevant bit:

$(document).ready(function() {
  var PusherLite = require("pusher_lite").default;

  var pusher = new PusherLite(window.app_key, {
    host: window.pusher_host,
    jwt: window.guardian_token,
    uniqueUserId: "robot" })
    // ssl: true - if you're connecting to my online server

  var publicChannel = pusher.subscribe("general")

  publicChannel.bind("new_message", function(payload) {
    var chat = $("#chat")
    chat.append("<p><strong>" + payload.name + "</strong> " + payload.message + "</p>");
    chat.scrollTop(chat.prop("scrollHeight"));
  })

  pusher.joinAll();

We can now continue in the same file with the Javascript that binds to the message input field, listening to the “Enter” key press event to send the messages:

  var message_element = $("#message");
  message_element.on('keypress', function(event) {
    if (event.keyCode != 13) { return; }

    var name_element    = $("#name");
    var check_element   = $("#channel");
    var payload = { name: name_element.val(), message: message_element.val() };

    if(!check_element.prop("checked")) {
      sendPusher(payload);
    } else {
      sendAPI(payload)
    }
    message_element.val('');
  });

  window.publicChannel = publicChannel;
})

And this is how we send messages to ExPusherLite, either directly through the full-duplex WebSockets:

function sendPusher(payload) {
  console.log("sending through socket")
  window.publicChannel.trigger('new_message', payload );
}

Or Posting to the available API:

function sendAPI(payload) {
  console.log("sending through API")
  $.ajax({
    type : 'POST',
    crossDomain: true,
    url : makeURL("new_message"),
    headers : { Authorization : 'Bearer ' + window.guardian_token },
    data : payload,
    success : function(response) {
      console.log(response);
      console.log("sent through API successfully");
    },
    error : function(xhr, status, error) {
      console.log(error);
    }
  });
}

function makeURL(event) {
  return "http://" + window.pusher_host + "/api/organizations/" + window.org_id + "/applications/" + window.app_key + "/event/" + event;
}

By the way, you can send messages using the API from the server-side if you want. Specifically from an ActiveJob process so you can keep your Rails web application fast, and you can use the opportunity to store the message in your database, or apply any filters.

And this is it! You now have a Rails application with WebSockets. You can have your lunch and eat it too.

If you want to see this example working, I published a demo app over at Heroku. It’s just a demo, it has no authentication, no cross-scripting sanitization, no nothing.

In summary: this is a Rails app (you could do it in Django, Laravel, ASP.NET MVC, it doesn’t matter) talking through WebSocket + APIs to a Phoenix cluster.

Next Steps

Keep following my blog (or my Twitter at @akitaonrails ) for more posts to come.

I am still considering if I will open the ExPusherLite code as open source, so let me know if you’re interested.

I am also considering if I will keep the current servers online as a cheap service. You can use it for free right now to play with it, but don’t use for production-level apps yet. As I am still heavily coding it, I will keep updating the servers, so there is no SLA. Let me know if you’re interested in such a service that keeps the code open source so you can trust it better.

There are important features still missing, such as proper SSL support, encrypted channels, better Presence tracking APIs and so on, but what’s available right now already covers most use cases for WebSockets.

And better: because this is Phoenix, because this is Elixir, and because this is Erlang, we get distributed PubSub for “free”. As I explained in my deployment post, this is a setup with a server in New York and another in London, just to showcase the distributed nature of Erlang.

It’s been very fun to play with Elixir for the past few days and how fast I was able to put together a full-featured solution like this. There were many puzzles that made me scratch my head, figuring out how to deal with cross origin issues, how to make the nodes find each other through the edeliver deployment, figuring out the missing bits in replacing exrm for distillery (which is a transition still taking place in the community), etc.

Now I am quite comfortable with the basics, from bootstrapping a project all the way to deploying in a cluster scenario. And I hope this service proves useful to more people.

As this is possibly my last post of the year: Happy New Year! And I will see you again in 2017!

For starters, Erlang does not need multiple processes like Ruby (which is limited to one connection per process, or per thread if you’re using a threaded-server like Puma). One single Erlang process will take over the entire machine, if you need to. Internally it will keep one real thread per machine-core. And each thread will have its own Scheduler to manage as many micro-processes as you need. You can read all about it in my post titled “Yocto Services”.

Moreover, Erlang has built-in capabilities to form a cluster, where each Erlang instance acts as a peer-to-peer Node, without the need for a centralized coordinator. You can read all about it in my post about Nodes. The power of Erlang is in how “easy” it is to form reliable distributed systems.

You can fire up many Phoenix instances and from one of the instances, it can broadcast messages to Users subscribed in Channels even if their sockets are connected to different instances. It’s seamless and you don’t need to do anything special in your code. Phoenix, Elixir and Erlang are doing all the heavy lifting for you behind the scenes.

No Heroku for You :-(

Because you want to take advantage of this scalability and high availability feature for distributed systems (in the small example of a real-time chat system) you will need to have more control over your infrastructure. This requirement rules out the majority of Platform as a Service (PaaS) offerings out there, such as Heroku. Heroku’s model revolves around single, volatile processes in isolated containers. Those jailed processes (dynos) are not aware of other processes or the internal networking, so you can’t fire up Dynos and have them form a cluster because they won’t be able to find each other.

If you already know how to configure Linux related stuff: Postgresql, HAproxy, etc, go ahead directly to the Phoenix-specific section.

IaaS (DigitalOcean) to the rescue!

You want long lived processes in network reachable servers (either through private networking, VPN, or plain simple - insecure! - public networks).

In this example I want to walk you through a very simple deployment using DigitalOcean (you can choose any IaaS, such as AWS, Google Cloud, Azure or whatever you feel more comfortable with).

I have created 4 droplets (all using the smallest size of 512Mb of RAM):

• 1 Postgresql database (single point of failure: it’s not the focus of this article to build a highly available, replicated database setup);
• 1 HAProxy server (single point of failure: again, it’s not the focus to create a highly available load balancing scheme);
• 2 Phoenix servers - one in the NYC datacenter and another in the London datacenter, to demonstrate how easy it is for Erlang to form clusters even with geographically separated boxes.

Basic Ubuntu 16.04 configuration

• Goals: configure locale, assure unattended updates are up, upgrade packages, install and configure Elixir and Node.
• You should also do: change SSH to another port and install [fail2ban](https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-fail2ban-on-ubuntu-14-04, disallow login through password).

You will want to read my post about configuring Ubuntu 16.04. To summarize, start by configuring proper UTF-8:

sudo locale-gen "en_US.UTF-8"
sudo dpkg-reconfigure locales
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
echo 'LC_ALL=en_US.UTF-8' | sudo tee -a /etc/environment
echo 'LANG=en_US.UTF-8' | sudo tee -a /etc/environment

Make sure you add a proper user into the sudo group and from now on do not use the root user. I will create a user named pusher and I will explain in another post why. You should create a username that suits your application.

adduser pusher
usermod -aG sudo pusher

Now log out and log in again through this user. ssh pusher@server-ip-address. If you’re on a Mac copy the public key of your SSH certificate like this:

ssh-copy-id -i ~/.ssh/id_ed25519.pub pusher@server-ip-address

It creates the .ssh/authorized_keys if it doesn’t exist, sets the correct permission bits and appends your public key. You can do it manually, of course.

DigitalOcean’s droplets start without a swap file and I’d recommend adding one, especially if you want to start with the smaller boxes with less than 1GB of RAM:

sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo cp /etc/fstab /etc/fstab.bak
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
sudo sysctl vm.swappiness=10
sudo sysctl vm.vfs_cache_pressure=50
echo 'vm.swappiness=10' | sudo tee -a /etc/sysctl.conf
echo 'vm.vfs_cache_pressure=50' | sudo tee -a /etc/sysctl.conf

Make sure you have unattended upgrades configured. You will want at least to have security updates automatically installed when available.

sudo apt install unattended-upgrades

Now, let’s install Elixir and Node (Phoenix needs Node.js):

wget https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb && sudo dpkg -i erlang-solutions_1.0_all.deb
curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential nodejs esl-erlang elixir erlang-eunit erlang-base-hipe
sudo npm install -g brunch
mix local.hex
mix archive.install https://github.com/phoenixframework/archives/raw/master/phoenix_new.ez # this is optional, install if you want to manually test phoenix in your box

Now you have an Elixir capable machine ready. Create a image snapshot over DigitalOcean, move it to the regions where you want to create your other droplets and use this image to create as many droplets as you need.

For this example, I created a second droplet in the London region, a third droplet for postgresql in the NYC1 region and a fourth droplet in the NYC3 region for HAProxy.

I will refer to their public IP addresses as “nyc-ip-address”, “lon-ip-address”, “pg-ip-address”, and “ha-ip-address”.

Basic PostgreSQL configuration

• Goal: basic configuration of Postgresql to allow the Phoenix servers to connect.
• To do: create a secondary role just to connect to the application database and another superuser role to create the database and migrate the schema. Also lock down the machine and configure SSH tunnels or another secure way, at least private networking, rather than allowing plain 5432 TCP port connections from the public Internet.

Now you can connect to ssh pusher@pg-ip-address and follow this:

sudo apt-get install postgresql postgresql-contrib

You should create a new role with the same name of the user you added above (“pusher” in our example):

$ sudo -u postgres createuser --interactive

Enter name of role to add: pusher
Shall the new role be a superuser? (y/n) y

$ sudo -u postgres createdb pusher

Postgresql expects to find a database with the same name as the role and the role should have the same name as the Linux user. Now you can use psql to set a password for this new role:

$ sudo -u postgres psql
\password pusher

Register a secure password, take note and let’s move on.

Postgresql comes locked down to external connections. One way to connect from the outside is to configure your servers to create an SSH tunnel to the database server and keep external TCP connections through port 5432 disavowed.

But for this example, we will just allow connections from the public Internet to the 5432 TCP port. Warning: this is VERY insecure!

Edit the /etc/postgresql/9.5/main/postgresql.conf and find the listen_addresses configuration line and allow it:

listen_addresses = '*'    # what IP address(es) to listen on;

This should bind the server to the TCP port. Now edit /etc/postgresql/9.5/main/pg_hba.conf and edit the end to look like this:

# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
host    all             all             your-local-machine-ip-address/24        trust
host    all             all             nyc-ip-address/24       trust
host    all             all             lon-ip-address/24       trust
# IPv6 local connections:
host    all             all             ::1/128                 trust

Save the configuration file and restart the server:

sudo service postgresql restart

See what I did there? I only allowed connections coming from the public IPs of the Phoenix servers. This does not make the server secure, just a little bit less vulnerable. If you’re behind a DHCP/NAT based network, just Google for “what’s my IP” to see your public facing IP address - which is probably shared by many other users, remember you’re allowing connections from an insecure IP to your database server! Once you make initial tests, create your new schema, then you should remove that your-local-machine-ip-address/24 line from the configuration.

From your Phoenix application, you can edit your local config/prod.secret.exs file to look like this:

# Configure your database
config :your_app_name, ExPusherLite.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: "pusher",
  password: "your-super-secure-pg-password",
  database: "your-app-database-name",
  hostname: "pg-ip-address",
  pool_size: 20

Replace the information for your server and database and now you can test it out like this:

MIX_ENV=prod iex -S mix phoenix.server

If you see a :econnrefused message from postgrex, then you’re in trouble. Re-check your configuration, restart the server and try again. If everything connects, you can run MIX_ENV=prod mix do ecto.create, ecto.migrate to prepare your database.

Finally, you will want to lock down the rest of your server with UFW, at the very least. UFW should come pre-installed in Ubuntu 16, so you can just do:

ufw allow 5432
ufw allow ssh
ufw enable

That’s it. And again, this does not make your server secure, it just makes it less insecure. There is a huge difference!

And by the way, if you’re a Docker fan:

DO NOT INSTALL A DATABASE INSIDE A DOCKER CONTAINER!

You have been warned!

Basic HAProxy configuration

• Goals: Provide a simple solution to load balance between our 2 Phoenix servers.
• To do: There is something wrong with session checking or something like that as sometimes I have to refresh my browser so I am not sent back to the login form in my application. Phoenix uses cookie-based sessions so I don’t think it is missing sessions.

Now let’s ssh pusher@ha-ip-address. This one is easy, let’s just install HAProxy:

sudo apt-get install haproxy

Edit /etc/haproxy/haproxy.cfg:

...
listen your-app-name
  bind 0.0.0.0:80
  mode http
  stats enable
  stats uri /haproxy?stats
  stats realm Strictly\ Private
  stats auth admin:some-secure-password-for-admin
  option forwardfor
  option http-server-close
  option httpclose
  balance roundrobin
  cookie SERVERID insert indirect nocache
  server us your_us_server_IP:8080 check cookie us1
  server uk your_uk_server_IP:8080 check cookie uk1

You can avoid the stats lines if you have other means of monitoring, otherwise set a secure password for the admin user. One very important part is the option http-server-close as explained in this other blog post, otherwise you may have trouble with Websockets.

For some reason I am having some trouble with my application after I login and it sets the session, sometimes I have to refresh to be sent to the correct page, not sure why yet and I believe it’s something in the HAProxy configuration. If anyone knows what it is, let me know in the comments section below. For now, I am just relying on Sticky sessions (server affinity) by making HAProxy write back a cookie with the server.

Now you can restart the server and enable UFW as well:

sudo service haproxy restart
sudo ufw allow http
sudo ufw allow https
sudo ufw allow ssh
sudo ufw enable

You can easily add SSL support to your application by configuring HAProxy (and not the Phoenix nodes). The DigitalOcean documentation is comprehensive, so just follow it. At the end, my haproxy.cfg looks like this:

global
   log /dev/log    local0
   log /dev/log    local1 notice
   chroot /var/lib/haproxy
   stats socket /run/haproxy/admin.sock mode 660 level admin
   stats timeout 30s
   user haproxy
   group haproxy
   daemon
   maxconn 2048
   tune.ssl.default-dh-param 2048

defaults
   log global
   mode http
   option httplog
   option dontlognull
   option redispatch
   option forwardfor
   option http-server-close
   timeout connect 5000
   timeout client  50000
   timeout server  50000

frontend www-http
   bind your_ha_proxy_IP:80
   reqadd X-Forwarded-Proto:\ http
   default_backend www-backend

frontend www-https
   bind your_ha_proxy_IP:443 ssl crt /etc/haproxy/certs/your_domain.pem
   reqadd X-Forwarded-Proto:\ https
   acl letsencrypt-acl path_beg /.well-known/acme-challenge/
   use_backend letsencrypt-backend if letsencrypt-acl
   default_backend www-backend

backend www-backend
   redirect scheme https if !{ ssl_fc }
   # setting session stickiness
   cookie SERVERID insert indirect nocache
   server us your_us_server_IP:8080 check cookie us1
   server uk your_uk_server_IP:8080 check cookie uk1

backend letsencrypt-backend
   server letsencrypt 127.0.0.1:54321

Finally, I will assume you have a DNS server/service somewhere where you can register the IP of this HAproxy server as an A record so you can access it by a full name such as “your-app-name.mydomain.com”.

Basic Phoenix Configuration

• Goal: configure the Phoenix app to be deployable. Configure the servers to have the necessary configuration files.
• To do: find out a way to cut down the super slow deployment times.

Finally, we have almost everything in place.

I will assume that you have a working Phoenix application already in place, otherwise create one from the many tutorials out there.

I have assembled this information from posts such as this very helpful one from Pivotal about an AWS-based deployment. In summary you must do a number of changes to your configuration.

When you’re developing your application, you will notice that whenever you run it, it delta-compiles what changed. The binary bits are in the _build/dev or _build/test in the form of .beam binaries (similar to what .class are for Java).

Different from Ruby or Python or PHP, you are not deploying source-code to production servers. It’s more akin to Java, where you must have everything compiled into binary bits and packaged into what’s called a release. It’s like a “.war” or “.ear” if you’re from Java.

To create this package people usually use “exrm”, but it’s being replaced by “distillery”, so we will use it.

Then, if you’re from Ruby you’re familiar with Capistrano. Or if you’re from Python, you know Capistrano’s clone, Fabric. Elixir has a similar tool (much simpler at this point), called “edeliver”. It’s your basic SSH automation tool.

You add them to mix.exs just like any other dependency:

...
def application do
  [mod: {ExPusherLite, []},
   applications: [..., :edeliver]]
end

defp deps do
  [...,
   {:edeliver, "~> 1.4.0"},
   {:distillery, "~> 1.0"}]
end
...

From the Pivotal blog post, the important thing to not forget is to edit this part in the config/prod.exs file:

http: [port: 8080],
url: [host: "your-app-name.yourdomain.com", port: 80],
...
config :phoenix, :serve_endpoints, true

You MUST hardcode the default PORT of the Phoenix web server and the allowed domain (remember the domain name you associated to your HAProxy server above? That one). And you MUST uncomment the :serve_endpoints, true line!

For edeliver to work you have to create a .deliver/config file like this:

USING_DISTILLERY=true

# change this to your app name:
APP="your-app-name"

# change this to your own servers IP and add as many as you want
US="nyc-ip-address"
UK="lon-ip-address"

# the user you created in your Ubuntu machines above
USER="pusher"

# which server do you want to build the first release?
BUILD_HOST=$US
BUILD_USER=$USER
BUILD_AT="/tmp/edeliver/$APP/builds"

# list the production servers declared above:
PRODUCTION_HOSTS="$US $UK"
PRODUCTION_USER=$USER
DELIVER_TO="/home/$USER"

# do not change here

LINK_VM_ARGS="/home/$USER/vm.args"

# For *Phoenix* projects, symlink prod.secret.exs to our tmp source
pre_erlang_get_and_update_deps() {
  local _prod_secret_path="/home/$USER/prod.secret.exs"
  if [ "$TARGET_MIX_ENV" = "prod" ]; then
    __sync_remote "
      ln -sfn '$_prod_secret_path' '$BUILD_AT/config/prod.secret.exs'

      cd '$BUILD_AT'

      mkdir -p priv/static

      mix deps.get

      npm install

      brunch build --production

      APP='$APP' MIX_ENV='$TARGET_MIX_ENV' $MIX_CMD phoenix.digest $SILENCE
    "
  fi
}

Remember the information we’ve been gathering since the beginning of this long recipe? These are the options you MUST change to your own. Just follow the comments in the file content above and add it to your git repository. By the way, your project is in a proper GIT repository, RIGHT??

If you like to use passphrase protected SSH private keys, then it’s going to be a huge pain to deploy because for each command, edeliver will issue an SSH command that will keep asking for you passphrase, a dozen times through everything. You’ve been warned! If you still don’t mind that, and you’re in a Mac you will have an extra trouble because the Terminal will not be able to create a prompt for you to input your passphrase. You must create an /usr/local/bin/ssh-askpass script:

#!/bin/bash
# Script: ssh-askpass
# Author: Mark Carver
# Created: 2011-09-14
# Licensed under GPL 3.0

# A ssh-askpass command for Mac OS X
# Based from author: Joseph Mocker, Sun Microsystems
# http://blogs.oracle.com/mock/entry/and_now_chicken_of_the

# To use this script:
#   Install this script running INSTALL as root
#
# If you plan on manually installing this script, please note that you will have
# to set the following variable for SSH to recognize where the script is located:
#   export SSH_ASKPASS="/path/to/ssh-askpass"

TITLE="${SSH_ASKPASS_TITLE:-SSH}";
TEXT="$(whoami)'s password:";
IFS=$(printf "\n");
CODE=("on GetCurrentApp()");
CODE=(${CODE[*]} "tell application \"System Events\" to get short name of first process whose frontmost is true");
CODE=(${CODE[*]} "end GetCurrentApp");
CODE=(${CODE[*]} "tell application GetCurrentApp()");
CODE=(${CODE[*]} "activate");
CODE=(${CODE[*]} "display dialog \"${@:-$TEXT}\" default answer \"\" with title \"${TITLE}\" with icon caution with hidden answer");
CODE=(${CODE[*]} "text returned of result");
CODE=(${CODE[*]} "end tell");
SCRIPT="/usr/bin/osascript"
for LINE in ${CODE[*]}; do
      SCRIPT="${SCRIPT} -e $(printf "%q" "${LINE}")";
done;
eval "${SCRIPT}";

Now do this:

sudo chmod +x /usr/local/bin/ssh-askpass
sudo ln -s /usr/local/bin/ssh-askpass /usr/X11R6/bin/ssh-askpass

Remember, Macs only. And now every time you try to deploy you will receive a number of graphical prompt windows asking for the SSH private key passphrase. It’s freaking annoying! And you must have XQuartz installed, by the way.

Now you must manually create 3 files in all Phoenix servers. Start with the your-app-name/vm.args:

-name us@nyc-ip-address
-setcookie @bCd&fG
-kernel inet_dist_listen_min 9100 inet_dist_listen_max 9155
-config /home/pusher/your-app-name.config
-smp auto

The /home/pusher/your-app-name is the directory where the release will be uncompressed after edeliver deploys the release tarball.

You must create this file in all Phoenix machines, by the way, changing the -name bit for the same name you declared in the .deliver/config file. The -setcookie should be any name, as long as it’s the same in all servers.

See the -config /home/pusher/your-app-name.config? Create that file with the following:

[{kernel,
  [
    {sync_nodes_optional, ['uk@lon-ip-address']},
    {sync_nodes_timeout, 30000}
  ]}
].

This is an Erlang source-code. On the NYC machine you must declare the London name, and vice-versa. If you have several machines, all of them but the one you’re in right now. Get it?

Finally, for the Phoenix app itself, you always have a config/prod.secret.exs that should never be git added to the repository, remember? This is where you put the Postgresql server information and random secret key to sign the session cookies:

use Mix.Config

config :your_app_name, YourAppName.Endpoint,
  secret_key_base: "..."

# Configure your database
config :your_app_name, YourAppName.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: "pusher",
  password: "your-super-secure-pg-password",
  database: "your-app-database-name",
  hostname: "pg-ip-address",
  pool_size: 20

# if you have Guardian, for example:
config :guardian, Guardian,
  secret_key: "..."

How do you create a new random secret key? From your development machine just run: mix phoenix.gen.secret and copy the generated string into the file above.

So now you must have those 3 files in each Phoenix server, in the /home/pusher home folder:

~/your-app-name/vm.args
~/prod.secret.exs
~/your-app-name.config

As per distillery documentation, you have to set an environment variable to let it know about the vm.args file, and this is SUPER important otherwise it will generate a default one that doesn’t set the proper name and cookie, so the nodes will not find each other after booting up.

Using sudo, edit the /etc/environment and add the line:

RELEASE_CONFIG_DIR=/home/pusher/your-app-name
VMARGS_PATH=/home/pusher/your-app-name/vm.args

You have to do it in all the phoenix servers.

In your local development directory you still have to run this:

mix release.init

It will generate a standard rel/config.exs file that you must add to your git repository with the following changes to the bottom:

...
environment :prod do
  plugin Releases.Plugin.LinkConfig
  ...
end

release :your_app_name do
  set version: current_version(:your_app_name)

  set applications: [
    your_app_name: :permanent
  ]
end

The plugin should enable the release to find the local vm.args file in the server (this is super important, otherwise the remote commands such as start, stop, ping, etc will not work and the nodes will not load the correct information to boot up and form a cluster).

Finally, all set, you can issue this command:

$ mix edeliver build release production --skip-mix-clean --verbose

If it finishes without errors you will have a message like the following in the end:

...
==> Release successfully built!
    You can run it in one of the following ways:
      Interactive: _build/prod/rel/your-app-name/bin/your-app-name console
      Foreground: _build/prod/rel/your-app-name/bin/your-app-name foreground
      Daemon: _build/prod/rel/your-app-name/bin/your-app-name start
--> Copying release 0.0.1 to local release store
--> Copying your-app-name.tar.gz to release store

RELEASE BUILD OF YOUR-APP-NAME WAS SUCCESSFUL!

Now, this will take an absurdly long time to deploy. That’s because it will git clone the source code of your app, fetch all Elixir dependencies (every time!), it will have to compile everything, then it will run the super slow npm install (every time!), brunch your assets, create the so-called “release”, tar and gzip it, download it and SCP it to the other machines you configured.

In the .deliver/config file you set a BUILD_HOST option. This is the machine where all this process takes place, so you will want to have at least this machine be beefier than the others. As I am using small 512Mb droplets, the process takes forever.

The last command will download the generated tarball. It has to do it to make sure you have the NIFs compiled in the native environment where they run, because if you use a Mac, binaries from Macs won’t run on Linux.

Now we must upload and decompress this tarball in each server with the following command:

mix edeliver deploy release to production

The slower your network, the longer this will take, as it’s uploading a big tarfile across the public internet, so make sure you’re in a fast connection. And when it finishes, we can restart the servers (if you already had instances running):

mix edeliver stop production
mix edeliver start production

If you do everything right, the edeliver process finishes without any error and it leaves a daemon running in your server, like this:

/home/pusher/your-app-name/erts-8.2/bin/beam -- -root /home/pusher/your-app-name -progname home/pusher/your-app-name/releases/0.0.1/your-app-name.sh -- -home /home/pusher -- -boot /home/pusher/your-app-name/releases/0.0.1/your-app-name -config /home/pusher/your-app-name/running-config/sys.config -boot_var ERTS_LIB_DIR /home/pusher/your-app-name/erts-8.2/../lib -pa /home/pusher/your-app-name/lib/your-app-name-0.0.1/consolidated -name us@nyc-ip-address -setcookie ex-push&r-l!te -kernel inet_dist_listen_min 9100 inet_dist_listen_max 9155 -config /home/pusher/your-app-name.config -mode embedded -user Elixir.IEx.CLI -extra --no-halt +iex -- console

It will still take a long time, but it should be easier. So this is a pro-tip for you, Linux users. Follow this Gist for more details, you must emulate what’s run from the .deliver/config file’s bottom half.

Also notice that I ran the migrations manually, but you can do it using mix edeliver migrate.

Read their documentation for more commands and configurations.

Also, do not forget to enable UFW:

sudo ufw allow ssh
sudo ufw allow 8080
sudo ufw allow 4369
sudo ufw allow proto tcp from any to any port 9100:9155
sudo ufw default allow outgoing
sudo ufw enable

Debugging Production bugs

Right after I deployed, obviously it failed. And the problem is that the /home/pusher/your-app-name/log/erlang.log files (they are automatically rotated so you may find several files ending in a number), you won’t see much.

What I recommend you to do is to change the config/prod.exs file ONLY in your development machine and change the log setting to config :logger, level: :debug, use the same prod.secret.exs you edited in the servers above and run it locally with MIX_ENV=prod iex -S mix phoenix.server.

For example, in development mode I had a code in the controller that was checking the existence of an optional query string parameter like this:

if params["some_parameter"] do
 ...

That was working fine in development but crashing in production, so I had to change it to:

if Map.has_key?(params, "some_parameter") do
 ...

Another thing was that Guardian was working normally in development, but in production I had to declare its application in the mix.exs like this:

def application do
  [mod: {ExPusherLite, []},
   applications: [..., :guardian, :edeliver]]
end

I was getting :econnrefused errors because I forgot to run MIX_ENV=prod mix do ecto.create, ecto.migrate as I instructed above. Once I figured those out, my application was up and running through the http://your-app-name.yourdomain.com, HAProxy was correctly forwarding to the 8080 port in the servers and everything runs fine, including the WebSocket connections.

Conclusion

As I mentioned above, this kind of procedure makes me really miss an easy to deploy solution such as Heroku.

The only problem I am facing right now is that when I log in through Coherence’s sign in page, I am not redirected to the correct URI I am trying ("/admin" in my case), sometimes reloading after sign in works, sometimes it doesn’t. Sometimes I am inside a “/admin” page but when I click one of the links it sends me back to the sign in page even though I am already signed in. I am not sure if it’s a bug in Coherence, ExAdmin, Phoenix itself or an HAProxy misconfiguration. I will update this post if I find out.

Edeliver also takes an obscene amount of time to deploy. Even waiting for sprockets to process in a git push heroku master deploy feels way faster in comparison. And this is for a very bare-bone Phoenix app. Having to fetch everything (because Hex doesn’t keep a local global cache, all dependencies are statically vendored in the project directory). Having to run the super slow npm doesn’t help either.

I still need to research if there are faster options, but for now what I have “works”.

And more importantly, now I have a scalable cluster for real-time bi-directional WebSockets, which is the main reason one might want to use Phoenix in the first place.

If you want to build a “normal” website, keep it simple and do it in Rails, Django, Express or whatever is your web framework of choice. If you want real-time communications the easy way, I might have a better solution. Keep an eye on my blog for news to come soon! ;-)

Cutting straight to the chase, many small Rails apps that I build start with 2 very simple add-ons: Devise for authentication and ActiveAdmin for basic database management. Then I build up from there.

Both Elixir and Phoenix are fast moving targets right now, which makes it difficult for a stable set of libraries to solidify properly, but I think that we’re finally getting past the early adopters curve.

One big point of contention has been user authentication. Many purists will argue that you need to build your own from scratch or by using low level libraries such as Guardian.

If you’re building an application that just exposes API endpoints, that’s probably fine. But for a full-featured web app meant for humans to use, this is hardly a good choice. I will not entertain the discussion today, as it’s beyond the point.

I am assuming that you at least followed both Elixir and Phoenix tutorials by now. If you didn’t go ahead and do it, it will take you one or two days to learn the very basics if you’re already an experienced rubyist. Then come back and read my posts on Elixir to understand where it stands out compared to everything else.

That being said, let’s get started.

Coherence (Devise alternative)

Finally, I found this project that’s been under heavy development for the past 6 months called Coherence. For all intents and purposes, it successfully mimics Devise in almost every way. And this is a very good thing for a lot of scenarios.

Their README is well explained enough so I will not copy and paste it here, just read it there to get up and running. But if you want to try out all of their features you can tweak their procedure with this set of options in the installation Mix task:

mix coherence.install --full --rememberable --invitable --trackable

Run mix help coherence.install to see a description for all the options.

And if you’re not tweaking the front-end, you can just add the proper sign up, sign in, sign out links by adding the following snippet to the web/templates/layout/app.html.eex:

<header class="header">
  <nav role="navigation">
    <ul class="nav nav-pills pull-right">
      <%= if Coherence.current_user(@conn) do %>
        <%= if @conn.assigns[:remembered] do %>
          <li style="color: red;">!!</li>
        <% end %>
      <% end %>
      <%= YourApp.Coherence.ViewHelpers.coherence_links(@conn, :layout) %>
      <li><a href="http://www.phoenixframework.org/docs">Get Started</a></li>
    </ul>
  </nav>
  
</header>
...

(By the way, whenever you see YourApp in the code snippets, you must change it to your app’s module name.)

If you get lost in their documentation you can check out their Coherence Demo repository for an example of a basic Phoenix app with Coherence already configured and working. You will mostly have to take care of web/router.ex to create a :protected pipeline and set the scopes accordingly.

If you do it correctly, this is what you will see:

It’s been a long while since I got excited by a simple sign in page!

Ex Admin (ActiveAdmin alternative)

Then, the next step I usually like to do is to add a simple Administration interface. To that end I found the Ex Admin, that’s been under heavy development since at least May of 2015. It’s so damn close to ActiveAdmin that its old theme will make you forget you’re not in a Rails application.

Again, it’s pretty straightforward to set it up by just following their README instructions.

Once you have it installed and configured, you can very quickly expose the User model into the Admin interface like this:

mix admin.gen.resource User

And we can edit the web/admin/user.ex with the following:

defmodule YourApp.ExAdmin.User do
  use ExAdmin.Register

  register_resource YourApp.User do
    index do
      selectable_column

      column :id
      column :name
      column :email
      column :last_sign_in_at
      column :last_sign_in_ip
      column :sign_in_count
    end

    show _user do
      attributes_table do
        row :id
        row :name
        row :email
        row :reset_password_token
        row :reset_password_sent_at
        row :locked_at
        row :unlock_token
        row :sign_in_count
        row :current_sign_in_at
        row :last_sign_in_at
        row :current_sign_in_ip
        row :last_sign_in_ip
      end
    end

    form user do
      inputs do
        input user, :name
        input user, :email
        input user, :password, type: :password
        input user, :password_confirmation, type: :password
      end
    end
  end
end

Yes, this is eerily similar to the ActiveAdmin DSL. Thumbs up for the team responsible, and it really shows how Elixir is well suited for Domain Specific Languages, if you’re into that.

If you followed the Coherence instructions, it asks you to add a :protected pipeline (a set of plugs) for your protected routes. For now you can add the /admin route to go through that pipeline. And for the uninitiated, a “plug” is similar in concept to a Rack app, or more specifically, a Rails middleware. But in Rails we only have one pipeline of middlewares. In Phoenix we can configure multiple pipelines for different sets of routes (browser and api, for example).

So we can add the following to web/router.ex:

...
scope "/admin", ExAdmin do
  pipe_through :protected
  admin_routes
end
...

With those simple contraptions in place, you will end up with something like this:

And if you’re still not convinced, how about changing to their old theme?

Hell yeah! Makes me feel right at home, although I really prefer the new theme. But you could replace your ActiveAdmin-based app for this one and your users would hardly notice the small differences in the interface. The behavior is basically the same.

If you still have questions on how to properly configure ExAdmin, check out their Contact Demo project, where you can find a real example.

Stitching a simple Admin role

Obviously, we don’t want to let all authenticated users access the Admin section.

So we can add a simple boolean field in the users table to indicate whether a user is an admin or not. You can change your migration to resemble this:

...
def change do
  create table(:users, primary_key: false) do

    add :name, :string
    add :email, :string
    ...
    add :admin, :boolean, default: false
    ...
  end
end
...

And you can configure the priv/repos/seeds.exs file to create 2 users, one admin and one guest:

YourApp.Repo.delete_all YourApp.User

YourApp.User.changeset(%YourApp.User{}, %{name: "Administrator", email: "admin@example.org", password: "password", password_confirmation: "password", admin: true})
|> YourApp.Repo.insert!

YourApp.User.changeset(%YourApp.User{}, %{name: "Guest", email: "guest@example.org", password: "password", password_confirmation: "password", admin: false})
|> YourApp.Repo.insert!

As this is just an exercise, you can drop the database and recreate it, like this: mix do ecto.drop, ecto.setup.

Coherence takes care of authentication, but we need to take care of authorization. You will find many examples online to something that resembles Rails’ Pundit, such as Bodyguard. But for this post I will stick to a simple Plug and create a new Router pipeline.

We need to create lib/your_app/plugs/authorized.ex and add the following:

defmodule YourApp.Plugs.Authorized do
  @behaviour Plug

  import Plug.Conn
  import Phoenix.Controller

  def init(default), do: default

  def call(%{assigns: %{current_user: current_user}} = conn, _) do
    if current_user.admin do
      conn
    else
      conn
        |> flash_and_redirect
    end
  end

  def call(conn, _) do
    conn
      |> flash_and_redirect
  end

  defp flash_and_redirect(conn) do
    conn
      |> put_flash(:error, "You do not have the proper authorization to do that")
      |> redirect(to: "/")
      |> halt
  end
end

Once a user signs in, Coherence puts the structure of the authenticated user into the conn (a Plug.Conn structure), so we can pattern match from it.

Now we need to create the router pipeline in the web/router.ex like this:

...
pipeline :protected_admin do
  plug :accepts, ["html"]
  plug :fetch_session
  plug :fetch_flash
  plug :protect_from_forgery
  plug :put_secure_browser_headers
  plug Coherence.Authentication.Session, protected: true
  plug YourApp.Plugs.Authorized
end
...
scope "/" do
  pipe_through :protected_admin
  coherence_routes :protected_admin
end
...
scope "/admin", ExAdmin do
  pipe_through :protected_admin
  admin_routes
end
...

The :protected_admin pipeline is exactly the same as :protected but we add the newly created YourApp.Plugs.Authorized plug at the end. And then we change the /admin scope to go through this new pipeline.

And that’s it. If you log in with the guest@example.org user, it will be kicked out to the homepage with a message saying that it’s not authorized. If you log in with the admin@example.org it will be able to access the ExAdmin interface in /admin.

Wrapping Up

Even though it’s now super simple to add Authentication, Administration and basic Authorization, don’t be fooled, the learning curve is still steep, even if you’ve been a Rails developer for a while.

Because of what’s underneath, the OTP architecture, the concepts of Applications, Supervisors, Workers, etc, it’s not immediately simple to wrap your head around what’s really going on. If you’re not careful, libraries such as Coherence or ExAdmin will make you feel like it’s just as simple as Rails.

And it’s not like that. Elixir is a completely different beast. And I mean it in a bad way, on the contrary. It’s meant for highly reliable and distributed systems and it demands way more knowledge, patience and training from the programmer.

On the other hand, exactly because libraries such as Coherence make it a lot easier to get started, you may become more motivated to put something up and running and then investing more time really understanding what’s going on underneath. So the recommendation is: get your hands dirty, get some quick instant gratification of seeing something running, and then go on and refine your knowledge. It will be way more rewarding if you do so.

I don’t see Phoenix as meant to just be a Rails replacement. This would be too easy. I see it more as another piece to make Elixir the best suited set of technologies to build highly scalable, highly reliable, highly distributed systems. Stopping at simple web applications would not fulfill Elixir’s potential.

If you don’t know what the “Pipe Operator” is in Elixir, take the following code:

Enum.sum(Enum.filter(Enum.map(1..100_000, &(&1 * 3)), odd?))

It’s ugly, we all know that. In an Object Oriented language like Ruby we would do something like this:

(1..100_000).
  map { |i| i * 3 }.
  select(&:odd?).
  reduce(&:+)

But Elixir does not have objects, only functions, so how can we code more elegantly? The solution came up in the form of the so called “Pipe Operator” which takes the last returning value and passes it through as the first argument of the next function call, like this:

1..100_000
  |> Stream.map(&(&1 * 3))
  |> Stream.filter(odd?)
  |> Enum.sum

So Ruby and Elixir “feel” the same when we are able to “chain” methods. In the Ruby world we don’t have the “need” for an operator like that. But it would be nice to have a mechanism that we could use to make our code more expressive, or more testable, or more readable. For example, what if we could write something like this:

(1..100_000).
  multiple_each_element_by_three.
  filter_out_odd_elements.
  give_the_sum_of_all_elements

Of course, this is a very constrained example with really bad method naming. But if we look at Mo Lawson’s article I linked above, it becomes more interesting:

keywords
  .map { |kw| kw.gsub(/(^[^[:alpha:]]+|[^[:alpha:]]+$)/, '') }
  .map { |kw| LegacySpanishCorrector.new.correct(kw) }
  .map { |kw| kw.gsub(/[^[:alpha:]\d'_\-\/]/, '') }
  .reject { |kw| STOP_WORDS.include?(kw) }

Ruby allows us to chain Enumerable methods one after the other, transforming the initial keywords list into “something” that is very difficult to infer just by looking at this code.

What about this other version?

class KeywordNormalizer
  def call(keywords)
    Collection.new(keywords)
      .strip_outer_punctuation
      .fix_spanish
      .clean_inner_punctuation
      .remove_stop_words
      .to_a
  end

  # ...
end

This is where he ends up by the end of his article: much more readable and each new isolated method is unit testable, resulting in more robust code.

The whole idea of this post is to introduce you to my new little gem “Chainable Methods”. The source code is on Github, as usual, so please contribute.

My gem will allow you to write Lawson’s last code like this:

KeywordNormalizer
  .chain_from(keywords)
  .strip_outer_punctuation
  .fix_spanish
  .clean_inner_punctuation
  .remove_stop_words
  .to_a
  .unwrap

You add the chainable_methods to your Gemfile as usual (you know the drill), then you can write Lawson’s module like this:

module KeywordNormalizer
  extend ChainableMethods
  
  def self.strip_outer_punctuation(array)
    array.map { |el| el.gsub(/(^[^[:alpha:]]+|[^[:alpha:]]+$)/, '') }
  end
  def self.fix_spanish(array)
    array.map { |el| LegacySpanishCorrector.new.correct(el) }
  end
  def self.clean_inner_punctuation(array)
    array.map { |el| el.gsub(/[^[:alpha:]\d'_\-\/]/, '') }
  end
  def self.remove_stop_words(array)
    array.reject { |el| STOP_WORDS.include?(el) }
  end
end

And that’s it, now you can chain everything like I showed previously. The pattern is:

1. Write a Module with class-level methods that receive at least one argument and extend the ‘ChainableMethods’ module:

module MyModule
  extend ChainableMethods

  def self.method_a(argument1)
    # ...
  end

  def self.method_b(argument1, argument2)
    # ...
  end

  def self.method_c(argument1)
    # yield a block passing the received argument
  end
end

2. Wrap an initial state that will be passed to the first method as its first argument:

some_initial_state = "Hello World"
MyModule
  .chain_from(some_initial_state)
  # ...

3. Chain as many methods from the module or methods that the returning state recognizes:

MyModule
  .chain_from(some_initial_state)
  .upcase
  .method_a
  .method_b(argument2)
  .method_c { |foo| foo }
  .split(" ")
  .join(", ")
  .unwrap

Notice that we do not need to pass the first argument to the methods within the ‘MyModule’ module, it will get the result from the last call automatically.

4. Do not forget to call #unwrap as the last call to get the last result from the chain.

An Experiment

And that’s it! I isolated this behavior only into modules that explicitly extend the ‘ChainableMethods’ module instead of automagically enabling it at the BasicObject level as many would initially think, because we don’t want a global ‘method_missing’ dangling around unchecked.

This behavior makes use of ‘method_missing’ so it’s not going to be fast in a synthetic benchmark against a direct method call, for obvious reasons. The purpose is not to be fast, just expressive. So keep that in mind.

The use case is: whenever you have some kind of transformation, you will want a chain of unit testable, isolated functions, and this is how you can get it without too much hassle.

This is an experiment. Because I’m using ‘method_missing’ there may be side-effects I am not seeing right now, so please let me know in the Github Issues and send feedback if it helped you out in some project.

Pull Requests are most welcome!

In Part 2 I will add the proper mechanisms to make a minimal core that is actually useful and deploy it to Heroku. In order to do that I need to implement the following:

• a simple Administration authentication (a hardcoded admin_username and admin_password will do for now)
• an admin restricted "/api/admin/apps" endpoint to manage new Applications. Each Application should have a randomly generated key and secret.
• the existing “/events” endpoint from Part 1 should be moved to "/api/apps/:app_id/events" and have access restricted to the authentication of the key and secret of the Application identified as “app_id”.
• for now, the Event will just broadcast to the appropriate topic. We want to be able to broadcast to everyone from an Application as well to specific topics within the Application scope.

As usual, the code for this section will be tagged as v0.2 in both the client demo and server-side Github repositories.

The App Resource

If this project of ours is to behave like Pusher.com, we need a way to create new “Applications”. Each client connecting to this service will be bound to this Application. Events should be restricted to the Application boundary. This is how we will isolate different clients connecting to the same server. So you can have one core serving several different web applications.

Once a new application is created, the client/consumer web app will have the pair of key and secret tokens that it will use to connect both the server-side triggers as well as the client-side Websocket listeners.

As a disclaimer, at this stage of development I will not implement any sophisticated authentication system such as OAuth2 or JWT. I will save this for posts to follow. For now I will use the Application’s key and secret just as simple username and password in an HTTP Basic Auth. This should be good enough for our purposes for the time being.

So, the very first step is to create such an “Application” resource and we can resort to Phoenix’s built-in JSON scaffold generator:

mix phoenix.gen.json App apps name:string slug:string key:string secret:string active:boolean

Most tutorials will show you the “phoenix.gen.html” generator, which behaves like Rails’s “scaffold”, generating HTML templates for each of the CRUD verbs. This is similar but it skips HTML and assumes this is going to be a JSON CRUD API.

We need to manually update the “web/router.ex” file like this:

# web/router.ex
scope "/api", ExPusherLite do
  pipe_through :api
  post "/apps/:app_slug/events", EventsController, :create
  scope "/admin" do
    resources "/apps", AppController, except: [:new, :edit]
  end
end

The “EventsController” is the one we implemented in Part 1 and that we will overhaul during this Part 2.

The generator gave us this new “AppController”, and similarly to Rails’ routes, the DSL is remarkably similar here. If you’re a Railer I bet you can instantly recognize the routes this DSL is generating.

The generator also created a proper migration for us:

# priv/repo/migrations/20151210131528_create_app.exs
defmodule ExPusherLite.Repo.Migrations.CreateApp do
  use Ecto.Migration

  def change do
    create table(:apps) do
      add :name, :string
      add :slug, :string
      add :key, :string
      add :secret, :string
      add :active, :boolean, default: false

      timestamps
    end
    create index(:apps, [:name], unique: true)
    create index(:apps, [:slug], unique: true)
  end
end

Again, remarkably similar to ActiveRecord’s Migration DSL. Migrations behave as you expect. You must run:

mix ecto.create # if you haven't already
mix ecto.migrate

This App resource will need the ability to create slugs out of the names (which we will use as “app_id”) and also generate random key and secret values. So we must add these dependencies to the “mix.exs” file:

# mix.exs
defp deps do
  [...,
   {:secure_random, "~> 0.2.0"},
   {:slugger, "~> 0.0.1"}]
end

The final App model is quite long, so I will break it down for you:

# web/models/app.ex
defmodule ExPusherLite.App do
  use ExPusherLite.Web, :model
  alias ExPusherLite.Repo

  schema "apps" do
    field :name, :string
    field :slug, :string
    field :key, :string
    field :secret, :string
    field :active, :boolean, default: true

    timestamps
  end

  @required_fields ~w(name)
  @optional_fields ~w()
  ...

This block declares the model Schema. Be careful if you generate a migration and then change its fields settings: you must remember to update the schema in the model. In my first attempt I didn’t include the “slug” field, so I rolled back the database migration (with “mix ecto.rollback”), changed the migration to add the “slug” field and re-ran the “ecto.migrate” task.

I was puzzled with the model not picking up the new field; after some time I remembered that Ecto models don’t attempt to fetch the real database schema and generate accessors dynamically, instead it relies on the explicitly declared schema block as shown above. After I added the new “slug” field in the schema block, then the model would properly use the new field.

# web/models/app.ex
...
def get_by_slug(slug) do
  Repo.get_by!(__MODULE__, slug: slug, active: true)
end
def hashed_secret(model) do
  Base.encode64("#{model.key}:#{model.secret}")
end
...

These are just helper functions to use in the AppController. The odd bit might be “MODULE” but this is just a shortcut for the atom representation of the current module, which is “ExPusherLite.App”. This is how you make a simple query to the model, it resembles Rails’ “App.get_by(slug: slug, active: true)”.

In Elixir convention, Ecto has functions with and without bangs ("get_by!" and “get_by”). If you want to catch an error you use the version without bangs and it will return either a “{:ok, result}” tuple or a “{:error, result}” and you can pattern match them. Or you can use the bang version and it will raise an exception. Depends on what you want to do.

# web/models/app.ex
...
def changeset(model, params \\ :empty) do
  model
  |> cast(params, @required_fields, @optional_fields)
  |> validate_length(:name, min: 5, max: 255)
  |> unique_constraint(:name)
  |> generate_key
  |> generate_secret
  |> slugify
  |> unique_constraint(:slug)
end
...

Second only to the Schema block I mentioned above, this “changeset/2” function is the most important part of a Model.

In Rails you just have the concept of a “Model” which is considered “Fat” because it deals with database operations, business logic and framework hooks all in the same place. In Phoenix you have to deal with at least 3 different concepts:

• You have a Repo, which receives a Changeset and uses it to insert or update the designated rows in the database table. You will see “Repo.get” or “Repo.insert”, not “App.find” or “App.save”.
• Then you have the Changeset which are just Elixir Maps (the ones with the syntax “%{key => value}”). It’s just a Hash, a Dictionary, a collection of key-value pairs. A Repo accepts Maps for its operations.
• Finally, you have the Model which actually give meaning and context to the raw Changesets. And the function “changeset/2” above is the one responsible for receiving a raw Map, piping it through a validation and transformation chain and return a valid Changeset for the Repo to use.

So, in a controller you will usually find code like this:

# web/controllers/app_controller.ex
...
def create(conn, %{"app" => app_params}) do
  changeset = App.changeset(%App{}, app_params)
  
  case Repo.insert(changeset) do
    {:ok, app} ->
      conn
      |> ...
    {:error, changeset} ->
      conn
      |> ...
  end
  ...

This is how you create a new, validated, changeset and then pass it to the Repo, treating the results in a pattern match block. Just compare the above changeset line with the beginning of the “changeset/2” function:

...
def changeset(model, params \\ :empty) do
  model
  |> cast(params, @required_fields, @optional_fields)
  |> validate_length(:name, min: 5, max: 255)
...

It maps the “%App{}” empty record to the “model” argument and the “app_params” that comes from the request (a map of the format “%{name => ‘foo’, active: => ’true’}”) to the argument “params”. Then it pipes the model and params to the “cast/4” function which will copy the values from the params map to the model map/changeset. And it keep passing the resulting changeset to the following functions, such as “validated_length/3” below, and so on. If the chain ends with no exceptions, you end up with a clean, validated changeset that you can just pass to the Repo to blindly insert to the database.

In the above implementation we are chaining filters to generate the key, secret and slug, and this is the implementation as private functions:

# web/models/app.ex
...
  defp generate_key(model) do
    if get_field(model, :key) do
      model
    else
      model
      |> put_change(:key, SecureRandom.uuid)
    end
  end

  defp generate_secret(model) do
    if get_field(model, :secret) do
      model
    else
      model
      |> put_change(:secret, SecureRandom.uuid)
    end
  end

  defp slugify(model) do
    if name = get_change(model, :name) do
      model
      |> put_change(:slug, Slugger.slugify_downcase(name))
    else
      model
    end
  end
end

The logic is set so new key/secret are generated only if the fields are empty and a new slug is generated only if the name has changes. And this is it, I told you the model code would be a bit large. You can see how to use the Slugger and SecureRandom libraries we added in the “mix.exs” before.

I also want to add the equivalent of a Rails seed file to create a test application so it’s easier for newcomers to know what to do. Phoenix has seeds and you can implement it like this:

# priv/repo/seeds.exs

alias ExPusherLite.App
alias ExPusherLite.Repo

# not using the App.changeset should just avoid all validations and generations
Repo.insert! %App{ slug: "test-app", name: "Test App", key: "test-app-fake-key", secret: "test-app-fake-secret", active: true }

Remember how I detailed the role of the “changeset/2” function in creating a clean and validated changeset, which is just a Map? You can skip that function altogether and hand craft your own final Map and pass it to the Repo. The Repo doesn’t care if this is a valid Map or not it will just try to insert it into the database regardless. And in this case the App Model prevents us from hardcoding keys and secrets, so this is how we do it in a seed file.

We can just run it directly like this:

mix run priv/repo/seeds.exs

The AppController just needs 2 changes. The first is to search the App through the slug field instead of the default ‘id’ field. This is simple enough, we just replace all calls to “app = Repo.get!(App, id)” to “app = App.get_by_slug(id)”, which is why we implemented this function in the model above.

The second thing is Authentication.

Adding Authentication

Now that we have an App model that can generate secure random UUIDs for key and secret, I will add a second level of authentication for administrators to be able to create such Apps.

For that I will just hard-code a secret in the config file of the application itself to serve as a development default. Like this:

# config/config.exs
...
config :ex_pusher_lite, :admin_authentication,
  username: "pusher_admin_username",
  password: "pusher_admin_password"
...
import_config "#{Mix.env}.exs"

You must add this block before the “import_config” function. Then you can override those values in the “config/prod.secret.exs” file, for example, like this:

# config/prod.secret.exs
...
config :ex_pusher_lite, :admin_authentication,
  username: "14e86e5fee3335fa88b0",
  password: "2b94ff0f07ce9769567f"

Of course, generate your own pair of secure username and password and replace it in the production environment if you intend to actually use this. For Heroku, we will still have to tweak this further, so keep this in mind.

Just to make the process easier, I also added the following helper function:

# lib/ex_pusher_lite.ex
...
  # Return this applicaton administration Basic HTTP Auth hash
  def admin_secret do
    admin_username = Application.get_env(:ex_pusher_lite, :admin_authentication)[:username]
    admin_password = Application.get_env(:ex_pusher_lite, :admin_authentication)[:password]
    secret = Base.encode64("#{admin_username}:#{admin_password}")
  end
end

This is how you fetch the configuration values. I am generating a simple Base64 encoded string out of the username concatenated with the password with a comma, which is what Basic HTTP Auth requires. I will use this admin hash for the “AppController” and each client must provide the key/secret in its own App instance to be able to trigger the “EventsController”.

For both controllers I will create a single Authentication Plug, like this:

# lib/ex_pusher_lite/authentication.ex
defmodule ExPusherLite.Authentication do
  import Plug.Conn

  alias ExPusherLite.App

  def init(assigns \\ [admin: false]), do: assigns

  def call(conn, assigns) do
    token =
      if assigns[:admin] do
        ExPusherLite.admin_secret
      else
        params = fetch_query_params(conn).params
        params["app_slug"] |> App.get_by_slug |> App.hashed_secret
      end

    "Basic " <> auth_token = hd(get_req_header(conn, "authorization"))
    if Plug.Crypto.secure_compare(auth_token, token) do
      conn
    else
      conn |> send_resp(401, "") |> halt
    end
  end
end 

As I explained in previous articles, a Plug is like a chainable Rails Middleware or even a Rack application. It must have a single “call/2” that receives a Plug.Conn structure and returns it back, allowing to form a chain/pipeline of Plugs.

We check if we want to compare with the Admin token or the App token and then retrieve the Basic HTTP authorization token that’s in the HTTP request connection structure (we retrieve individual header values through the “get_req_header/2” function). Finally we make a secure compare between the tokens.

To enable this plug in the controllers we just add it like this:

# web/controllers/app_controller.ex
defmodule ExPusherLite.AppController do
  use ExPusherLite.Web, :controller

  alias ExPusherLite.App
  plug ExPusherLite.Authentication, [admin: true]
  ...

 defmodule ExPusherLite.EventsController do
   use ExPusherLite.Web, :controller

-  plug :authenticate
+  plug ExPusherLite.Authentication
   ...

In Part 1 we had a simpler “plug :authenticate” in the EventsController. We can remove it and also the “authenticate/2” function. We just refactored it into a better function that also serves administration authentication now, but the idea is the same.

This is it: the basics for API authentication. Again, this is not the best solution as the username/password pair goes in the URL and it’s open to man-in-the-middle attacks. SSL only encrypts the HTTP body but the URL is still open.

For example, if an administrator wants to create a new application, he must do the following:

curl --data "app[name]=foo-app" http://pusher_admin_username:pusher_admin_password@localhost:4000/api/admin/apps

And this would be one example of the resulting JSON representation of the new app:

{"data":{"slug":"foo-app","secret":"8ef69064-0d7e-c9ef-ac14-b6b1db303e7a","name":"foo-app","key":"9400ad21-eed8-117a-bce5-845262e0a09e","id":5,"active":true}}%

With this new key and secret in hand, we can update our client demo to make use of the new app.

Configuring the Client Demo

We must start by adding the proper Application details in the “.env” file:

PUSHER_URL: "localhost:4000"
PUSHER_APP_ID: "foo-app"
PUSHER_KEY: "9400ad21-eed8-117a-bce5-845262e0a09e"
PUSHER_SECRET: "8ef69064-0d7e-c9ef-ac14-b6b1db303e7a"
PUSHER_CHANNEL: "foo-topic"

We must also tweak the “config/secrets.yml” to reflect the new metadata (development, test, and production must follow this):

development:
  secret_key_base: ded7c4a2a298c1b620e462b50c9ca6ccb60130e27968357e76cab73de9858f14556a26df885c8aa5004d0a7ca79c0438e618557275bdb28ba67a0ffb0c268056
  pusher_url: <%= ENV['PUSHER_URL'] %>
  pusher_app_id: <%= ENV['PUSHER_APP_ID'] %>
  pusher_key: <%= ENV['PUSHER_KEY'] %>
  pusher_secret: <%= ENV['PUSHER_SECRET'] %>
  pusher_channel: <%= ENV['PUSHER_CHANNEL'] %>
  ...

And we can create an initializer to make it easier to use this metadata properly:

# config/initializers/pusher_lite.rb
module PusherLite
  def self.uri
    key    = Rails.application.secrets.pusher_key
    secret = Rails.application.secrets.pusher_secret
    app_id = Rails.application.secrets.pusher_app_id
    url    = Rails.application.secrets.pusher_url

    uri = "http://#{key}:#{secret}@#{url}/api/apps/#{app_id}/events"
    URI.parse(uri)
  end
end 

Again, the Rails app will trigger the ExPusherLite server using the Basic HTTP Auth. Do not be fooled into thinking this is “secure”, it just “feels a bit secure through obscurity”. You have been warned, wait for the next articles on this subject. But this is usable in controlled environments.

To finalize the upgrades, we must change the client-side access to the new metadata, first changing the application layout:

<!-- app/views/layouts/application.html.erb -->
...
+  <meta name="pusher_host" content="<%= Rails.application.secrets.pusher_url %>">
-  <meta name="pusher_key" content="<%= Rails.application.secrets.pusher_key %>">
+  <meta name="pusher_app_id" content="<%= Rails.application.secrets.pusher_app_id %>">
   <meta name="pusher_channel" content="<%= Rails.application.secrets.pusher_channel %>">
...

The javascript “index.es6” fetches from this meta headers, so we must change them there:

# app/assets/javascripts/application/pages/home/index.es6
...
     let guardianToken = $("meta[name=guardian-token]").attr("content")
     let csrfToken     = $("meta[name=guardian-csrf]").attr("content")
 
+    let pusherHost    = $("meta[name=pusher_host]").attr("content")
-    let pusherKey     = $("meta[name=pusher_key]").attr("content")
+    let pusherApp     = $("meta[name=pusher_app_id]").attr("content")
     let pusherChannel = $("meta[name=pusher_channel]").attr("content")
 
-    let socket = new Socket("ws://localhost:4000/socket", {
+    let socket = new Socket(`ws://${pusherHost}/socket`, {
       params: { guardian_token: guardianToken, csrf_token: csrfToken }
     })
     socket.connect()
 
     // Now that you are connected, you can join channels with a topic:
-    let channel = socket.channel(pusherChannel, {})
+    let channel = socket.channel(`public:${pusherApp}`, {})
     channel.join()
       .receive("ok", resp => { console.log("Joined successfully", resp) })
       .receive("error", resp => { console.log("Unable to join", resp) })
 
-    channel.on("msg", data => {
+    channel.on(`${pusherChannel}:msg`, data => {
       let new_line = `<p><strong>${data.name}<strong>: ${data.message}</p>`
       $(".message-receiver").append(new_line)
     })
+
+    channel.on("msg", data => {
+      let new_line = `<p><strong>Broadcast to all channels</strong>: ${data.message}</p>`
+      $(".message-receiver").append(new_line)
+    })
   }

One important modification from Part 1 is that the WebSocket host was hardcoded to “localhost” and here we are making it configurable through the meta tags. Right now, for localhost tests, we are using the plain “ws://” protocol but when we deploy to Heroku we will change it to “wss://” for SSL. Same thing for the “PusherLite” initializer. Keep that in mind.

Now it’s subscribing to a different format of topic/channel. In Part 1 it would be something like: “public:test_channel” now we are listening to “public:foo-app”, so the application is the Websocket subscription “topic”.

Then we are changing the socket listener to listen for 2 different events. The first one is in the format “test_channel:msg”. So this is how we must now send messages to a specific “channel” within an “app/topic”.

And last we still listen to the old “msg” event, but this serves as a “broadcast” event for all connected clients subscribed to this particular “foo-app” Application. Now web clients can listen to specific “channels” within the “app” but also receive system wide “broadcast” messages. This is a big improvement and it didn’t require much on the Javascript side.

But what more does it take to make this “channel-only and broadcast” system work? First, we start changing the web form to allow a user to choose between sending a channel-only message or a broadcast, like this:

<!-- app/views/home/index.html.erb -->
...
     <%= f.text_field :name, placeholder: "Name" %>
     <%= f.text_field :message, placeholder: "Message" %>
+    <%= f.check_box :broadcast %>
     <%= f.submit "Send message", class: "pure-button pure-button-primary" %>
   </fieldset>
...

Now the EventsController must accept this new parameter:

# app/controllers/events_controller.rb
...
  def event_params
    params.require(:pusher_event).permit(:name, :message, :broadcast)
  end
end

Finally, the Model must use this new information before posting to the ExPusherLite server:

# app/models/pusher_event.rb
class PusherEvent
  include ActiveModel::Model

  attr_accessor :name, :message, :broadcast
  validates :name, :message, presence: true

  def save
    topic = if broadcast == "1"
              "#general"
            else
              Rails.application.secrets.pusher_channel
            end

    Net::HTTP.post_form(PusherLite.uri, {
      "topic" => topic,
      "event" => "msg",
      "scope" => "public",
      "payload" => {"name" => name, "message" => message}.to_json
    })
  end
end

I am just assuming a hard-coded “#general” string to serve as the broadcast trigger for the server. Now we must make the server accept this new protocol schema, so let’s go back to Elixir.

First we must start with the counterpart for the previous POST trigger, ExPusherLite.EventsController:

# web/controllers/events_controller.ex
 defmodule ExPusherLite.EventsController do
   use ExPusherLite.Web, :controller
 
-  plug :authenticate
+  plug ExPusherLite.Authentication
 
-  def create(conn, params) do
-    topic = params["topic"]
-    event = params["event"]
+  def create(conn, %{"app_slug" => app_slug, "event" => event, "topic" => topic, "scope" => scope} = params) do
     message = (params["payload"] || "{}") |> Poison.decode!
-    ExPusherLite.Endpoint.broadcast! topic, event, message
+    topic_event =
+      if topic == "#general" do
+        event
+      else
+        "#{topic}:#{event}"
+      end
+    ExPusherLite.Endpoint.broadcast! "#{scope}:#{app_slug}", topic_event, message
     json conn, %{}
   end
   ...

The first difference is that I am pattern matching from the arguments directly to the “topic” and “event” variables. This function is also aware of the “#general” string the client can send to indicate an app-wide broadcast. And the new topic is the concatenation of “topic” and “event” to allow for “channel-only” messages.

To connect this all to the WebSocket handler, we must make the following changes:

# web/channels/room_channel.ex
-  def handle_in("msg", payload, socket = %{ topic: "public:" <> _ }) do
-    broadcast socket, "msg", payload
+  def handle_in(topic_event, payload, socket = %{ topic: "public:" <> _ }) do
+    broadcast socket, topic_event, payload
     { :noreply, socket }
   end
 
-  def handle_in("msg", payload, socket) do
+  def handle_in(topic_event, payload, socket) do
     claims = Guardian.Channel.claims(socket)
     if permitted_topic?(claims[:publish], socket.topic) do
-      broadcast socket, "msg", payload
+      broadcast socket, topic_event, payload
       { :noreply, socket }
   ...

Now, the Channel does not pattern match on a specific event, it lets it through without further validation, trusting that the EventsController is doing the right thing. I will come back to this piece for improvements in the future, possibly.

Deploying our first Phoenix app to Heroku!

In this section we will just follow the official documentation, so read it if you want more details.

Let’s get started:

heroku apps:create your-expusherlite --buildpack "https://github.com/HashNuke/heroku-buildpack-elixir.git"
heroku buildpacks:add https://github.com/gjaldon/heroku-buildpack-phoenix-static.git

I am naming the application “your-expusherlite” but you should change it to your own name, of course. And the rest of the configuration data are all examples that you must change for your own needs.

Heroku relies on environment variables. So we start by erasing “config/prod.secret.exs” and change “config/prod.exs” to look like this:

config :ex_pusher_lite, ExPusherLite.Endpoint,
  http: [port: {:system, "PORT"}],
  url: [scheme: "https", host: "your-expusherlite.herokuapp.com", port: 443], force_ssl: [rewrite_on: [:x_forwarded_proto]],
  cache_static_manifest: "priv/static/manifest.json",
  secret_key_base: System.get_env("SECRET_KEY_BASE")

# Configure your database
config :ex_pusher_lite, ExPusherLite.Repo,
  adapter: Ecto.Adapters.Postgres,
  url: System.get_env("DATABASE_URL"),
  pool_size: 20

config :ex_pusher_lite, :admin_authentication,
  username: System.get_env("PUSHER_ADMIN_USERNAME"),
  password: System.get_env("PUSHER_ADMIN_PASSWORD")

# remove this line:
# import_config "prod.secret.exs"

Now we must configure the environment variables “SECRET_KEY_BASE”, “PUSHER_ADMIN_USERNAME” and “PUSHER_ADMIN_PASSWORD”. Use the included “mix phoenix.gen.secret” to generate those.

heroku config:set SECRET_KEY_BASE="`mix phoenix.gen.secret`"
heroku config:set PUSHER_ADMIN_USERNAME="FPO0QUkqbAP6EGjElqBzDQuMs8bhFS3"
heroku config:set PUSHER_ADMIN_PASSWORD="n78DPGmK3DBQy8YAVyshiGqcXjjSXSD"

Then it’s just a matter of waiting for the good old “git push heroku master” to finish compiling everything for the first time. And because this is the first deploy you should not forget to run “heroku run mix ecto.migrate” to create the database table.

Now, if I did everything right, as an Administrator that knows the above hardcoded secrets I should be able to create a new Application like this:

curl --data "app[name]=shiny-new-app" https://FPO0QUkqbAP6EGjElqBzDQuMs8bhFS3:n78DPGmK3DBQy8YAVyshiGqcXjjSXSD@your-expusherlite.herokuapp.com/api/admin/apps

And this is the result I got!

{"data":{"slug":"shiny-new-app","secret":"42560373-0fe1-506e-28ca-35ab5221fb3d","name":"shiny-new-app","key":"958c16e7-ab93-dac0-0fc6-6cb864e26358","id":1,"active":true}}

Great, now that we have a valid Application key and secret we can configure our Rails Client Demo and deploy it to Heroku as well.

Deploying the Rails Client to Heroku

This is a simple Rails application, we can just create the app and deploy right away:

heroku create your-expusherlite-client
heroku config:set PUSHER_URL=your-expusherlite.herokuapp.com
heroku config:set PUSHER_APP_ID=shiny-new-app
heroku config:set PUSHER_KEY=958c16e7-ab93-dac0-0fc6-6cb864e26358
heroku config:set PUSHER_SECRET=42560373-0fe1-506e-28ca-35ab5221fb3d
heroku config:set PUSHER_CHANNEL=shiny-new-topic
git push heroku master

I’m assuming the readers of this post already know how to configure a Rails app properly for Heroku. Just to mention it, I configure this app with the 12 factor and Puma gems and added a proper Procfile. Another very small change was changing the “pusher_lite.rb” initializer to create a URI with “https” because the ExPusherLite we deployed to production requires SSL by default.

There is one more caveat. Being led by experienced web programmers, they made sure that, unlike this bare-bones exercise here, the Phoenix framework itself is secure. One such example is to disallow Websocket connections from different hosts.

Out of the box, the “phoenix.js” Socket will fail connection when we try to connect from the “your-expusherlite-client.herokuapp.com” Rails app host to the Phoenix app in “your-expusherlite.herokuapp.com” with the following error:

WebSocket connection to 'wss://your-expusherlite.herokuapp.com/socket/websocket?guardian_token=N_YCG6hGK7…iOlsicHVibGljOioiXX0._j6s2LiaKde9rBhnTMxDkm0XV5u89pNh1AdLFY6Rlt8&vsn=1.0.0' failed: Error during WebSocket handshake: Unexpected response code: 403

And in the Phoenix log we will see this very helpful message:

[error] Could not check origin for Phoenix.Socket transport.

This happens when you are attempting a socket connection to
a different host than the one configured in your config/
files. For example, in development the host is configured
to "localhost" but you may be trying to access it from
"127.0.0.1". To fix this issue, you may either:

  1. update [url: [host: ...]] to your actual host in the
     config file for your current environment (recommended)

  2. pass the :check_origin option when configuring your
     endpoint or when configuring the transport in your
     UserSocket module, explicitly outlining which origins
     are allowed:

        check_origin: ["https://example.com",
                       "//another.com:888", "//other.com"]

Unless you know what a Cross-Site Web Socket Hijacking is, you will prefer to keep the default settings as they are. In a green-field Phoenix app, the Web part will connect to the Web Socket in the same app and, therefore, in the same host, so this is not an issue.

In this case I am making a separate micro-service to mimic Pusher.com behavior so it should be able to accept Web Socket connections from different hosts.

If you control the applications being created, you will likely prefer to make the “check_origin” setting read from your database for the exact hosts. As a feature for next time I could add a “host” field in the “App” model and use it to validate connections in the transport configuration. For the time being I will just make it accept any hosts:

# web/channels/user_socket.ex
defmodule ExPusherLite.UserSocket do
  use Phoenix.Socket

  ## Channels
  channel "*", ExPusherLite.RoomChannel

  ## Transports
  transport :websocket, Phoenix.Transports.WebSocket, check_origin: false
  ...

And this is it! Now the Rails app should be able to connect and send messages! And you should be able to create any number of new apps and connect all of them to this same service.

Conclusion

Right now, we have a functional, albeit bare-bones, Pusher.com clone that will work for any number of use cases where Pusher.com would be used.

As I warned many times before, the security part is still flaky and needs work. I will still extend on what Daniel began with Guardian to authenticate Web Socket users to private channels as well. And the core should also receive auditing and reporting capabilities (to be able to report usage, number of active connections, throughput of events, keep at least a short history of events so new connections can retrieve the last sent messages, and so on).

But from here it’s a matter of adding features to an already working core. And this is nothing more than Phoenix out-of-the-box without too much added on top of it! It says a lot about the current state of maturity of this very capable framework.

In terms of performance, for this very simple example, I spun up 1 free Heroku dyno for each app.

The Rails app is able to respond to the front-end user interface in around 2ms. And the Sucker Punch job - which does the heavy HTTP POST to ExPusherLite - takes in the order of 30ms or less.

The Phoenix server receives the HTTP POST and performs the broadcast in around less than 6ms. Also quite fast. The times will vary a lot because I believe the free dyno is not only slow but also stays in highly shared metal boxes, getting impact from other neighbor apps running in the same box.

Because we already have an administrative API to create and manage apps (create new ones, delete, update, etc) we can already create a separate application in any other framework to make a dashboard for admins or for a self-serving front-end for developers to register new apps and receive key/secret pair to add in their own applications.

Both the ExPusherLite server and the demo client are deployed in Heroku and you can test the client right now clicking here. The admin keys are different from what I showed in this post, of course, so you won’t be able to create new apps, but you can deploy it yourself to your own environment.

As a Rails developer there are a few things we can’t do in Rails. One of them is to deal with real-time messaging.

Rails 5 will bring Action Cable, and it might be good enough for most cases. It uses Faye, which in turn is based on Eventmachine. You can implement a good enough solution for Websockets using Faye in your Rails 4.2 app right now.

Another option is to avoid the trouble altogether and use a messaging service. One option I always recommend for zero friction is to use Pusher.com.

The Basics

You will want to clone from my example repository, like this:

git clone https://github.com/akitaonrails/pusher_lite_demo
cd pusher_lite_demo
git checkout tags/v0.1 -b v0.1
bundle

This is a very very simple implementation of a real-time, websocket-based, chat using Pusher. The idea goes like this:

We start by having a front-end Form to send messages

<!-- app/views/home/index.html.erb -->
<%= form_for @event, url: events_path, remote: true, html: {class: "pure-form pure-form-stacked"} do |f| %>
  <fieldset>
    <legend>Send your message remotely</legend>
    <%= f.text_field :name, placeholder: "Name" %>
    <%= f.text_field :message, placeholder: "Message" %>
    <%= f.submit "Send message", class: "pure-button pure-button-primary" %>
  </fieldset>
<% end %>

It’s using Rails built-in jQuery support for Ajax posting the form to the “EventsController#create” method:

# app/controllers/events_controller.rb
class EventsController < ApplicationController
  def create
    SendEventsJob.perform_later(event_params)
  end

  def event_params
    params.require(:pusher_event).permit(:name, :message)
  end
end

Just to annotate the process, the “routes.rb” looks like this:

# config/routes.rb
Rails.application.routes.draw do
  resources :events, only: [:create]

  root 'home#index'
end

The HTML layout looks like this:

<!-- app/views/layout/application.html.erb -->
<!DOCTYPE html>
<html>
<head>
  <title>Pusher Lite Demo</title>
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="pusher_key" content="<%= Rails.application.secrets.pusher_key %>">
  <meta name="pusher_channel" content="<%= Rails.application.secrets.pusher_channel %>">
  <%= stylesheet_link_tag    'application', media: 'all', 'data-turbolinks-track' => true %>
  <%= javascript_include_tag 'application', 'data-turbolinks-track' => true %>
  <script src="https://akitaonrails.github.io//js.pusher.com/3.0/pusher.min.js"></script>
  <%= csrf_meta_tags %>
</head>
<body>

<div class="pure-menu pure-menu-horizontal">
    Pusher Client Demo
...
</div>

<div class="pure-g-r">
  <div class="pure-u-1-3 message-form">
    <%= yield %>
  </div>
  
  <div class="pure-u-1-3 message-receiver">
  </div>
</div>

</body>
</html>

This layout imports the default “application.js” which configures Pusher, establishes the Websocket connection and subscribes to messages on a specific topic with specific events:

// app/assets/javascript/application.js
//= require jquery
//= require jquery_ujs
//= require turbolinks
//= require_tree .

$(document).on("page:change", function(){
  var pusherKey = $("meta[name=pusher_key]").attr("content");
  var pusher = new Pusher(pusherKey, { encrypted: true });

  var pusherChannel = $("meta[name=pusher_channel]").attr("content");
  var channel = pusher.subscribe(pusherChannel);
  
  channel.bind('new_message', function(data) {
    var new_line = "<p><strong>" + data.name + "<strong>: " + data.message + "</p>";
    $(".message-receiver").append(new_line);
  });
});

It gets the configuration metadata from the layout meta tags which grabs the values from “config/secrets.yml”:

development:
  secret_key_base: ded7c4a2a298c1b620e462b50c9ca6ccb60130e27968357e76cab73de9858f14556a26df885c8aa5004d0a7ca79c0438e618557275bdb28ba67a0ffb0c268056
  pusher_url: <%= ENV['PUSHER_URL'] %>
  pusher_key: <%= ENV['PUSHER_KEY'] %>
  pusher_channel: test_chat_channel

test:
  secret_key_base: f51ff494801ff0f9e1711036ef6f2f6f1e13544b02326adc5629c6833ae90f1a476747fae94b792eba8a444305df8e7a5ad53f05ea4234692ac96cc44f372029
  pusher_url: <%= ENV['PUSHER_URL'] %>
  pusher_key: <%= ENV['PUSHER_KEY'] %>
  pusher_channel: test_chat_channel

# Do not keep production secrets in the repository,
# instead read values from the environment.
production:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
  pusher_url: <%= ENV['PUSHER_URL'] %>
  pusher_key: <%= ENV['PUSHER_KEY'] %>
  pusher_channel: <%= ENV['PUSHER_CHANNEL'] %>

And as I’m using dotenv-rails, the “.env” looks like this:

PUSHER_URL: "https://14e86e5fee3335fa88b0:2b94ff0f07ce9769567f@api.pusherapp.com/apps/159621"
PUSHER_KEY: "14e86e5fee3335fa88b0"
PUSHER_CHANNEL: "test_chat_channel"

Pusher is configured on the server-side through this initializer:

# config/initializers/pusher.rb
require 'pusher'

Pusher.url = Rails.application.secrets.pusher_url
Pusher.logger = Rails.logger

Finally, the “EventsController#create” actually does an async call to a SuckerPunch job:

class SendEventsJob < ActiveJob::Base
  queue_as :default

  def perform(event_params)
    @event = PusherEvent.new(event_params)
    @event.save
  end
end

By the way, as a segue, SuckerPunch is a terrific solution for in-process asynchronous tasks. It’s a better option to start with it without having to implement a separate system with Sidekiq workers.

Once you have larger job queues or jobs that are taking too long, then go to Sidekiq. If you use ActiveJob, the transition is as simple as changing the following configuration line in the “config/application.rb” file:

config.active_job.queue_adapter = :sucker_punch

This job just calls the “save” method in the fake-model “PusherEvent”:

class PusherEvent
  include ActiveModel::Model

  attr_accessor :name, :message
  validates :name, :message, presence: true

  def save
    Pusher.trigger(Rails.application.secrets.pusher_channel, 'new_message', {
      name: name,
      message: message
    })
  end
end

As it’s very simple, the Gemfile is equally simple:

gem 'pusher'
gem 'dotenv-rails'
gem 'purecss-rails'
gem 'sucker_punch'

So what it does is very simple:

1. The application loads Pusher and the necessary configuration.
2. When the user goes to “http://localhost:3000” he is presented with the message Form.
3. When the user posts the message, it ends in “EventsController#create”, which calls SuckerPunch’s “SendEventsJob”, instantiates a new “PusherEvent” model with the received form params and finally triggers the message to the Pusher server.
4. The same form page also loads the Pusher javascript client and connects to the “test_chat_channel” topic and listens to the “new_message” event, which is the same thing the “Pusher.trigger” call sends together with the form message params.
5. The Pusher server broadcasts the received trigger message to all subscribed Websocket clients.
6. The Pusher javascript client in the user’s browser receives the new message and just formats the message and appends to the “message-receiver” div HTML block in the same page.

Pusher has support for authenticated users, private channels and more, but this is 80% of the usage for most cases. You can implement this as a chat system, a notification system, or anything like that.

Your Rails app sets up the front-end HTML/Javascript to connect to Pusher, listening to certain topics and events and the same Rails app triggers Pusher on the server-side, posting new messages. Pusher receives messages and broadcasts to the clients that subscribe to its topics. That’s it.

Ex Pusher Lite - Part 1: Initial Phoenix-based Pusher replacement

My original idea was to make a drop-in replacement for the Pusher server, using the same Pusher client, but for now it was not easy to do so.

Instead, this Part 1 will focus on implementing an initial ExPusherLite server that also receives events triggered by the same Rails server-side controller process and broadcasting to the same Rails front-end component through WebSockets.

I followed Daniel Neighman’s tutorial. I had to do a few adjustments to have it working (and as this is still Part 1, it’s not a complete solution yet!)

You can clone the initial version from my other Github repository like this:

git clone https://github.com/akitaonrails/ex_pusher_lite
cd ex_pusher_lite
mix deps.get

The tutorial implemented initial setup for Guardian and Joken for JSON Web Tokens. I am still getting used to how channels are implemented in Phoenix.

It already comes pre-configured with a single socket handler that multiplexes connections. You start through the EndPoint OTP application:

# lib/ex_pusher_lite/endpoint.ex
defmodule ExPusherLite.Endpoint do
  use Phoenix.Endpoint, otp_app: :ex_pusher_lite

  socket "/socket", ExPusherLite.UserSocket
  ...

This application is started by the main supervisor in “lib/ex_pusher_lite.ex”. It points the endpoint “/socket” to the socket handler “UserSocket”:

# web/channels/user_socket.ex
defmodule ExPusherLite.UserSocket do
  use Phoenix.Socket

  ## Channels
  channel "*", ExPusherLite.RoomChannel

  ## Transports
  transport :websocket, Phoenix.Transports.WebSocket
  # transport :longpoll, Phoenix.Transports.LongPoll
  ...

The “channel” function comes commented out, so I started by uncommenting it. You can pattern match the topic name like “public:*” to different Channel handlers. For this simple initial test I am sending everything to the “RoomChannel”, which I had to create:

defmodule ExPusherLite.RoomChannel do
  use Phoenix.Channel
  use Guardian.Channel

  # no auth is needed for public topics
  def join("public:" <> _topic_id, _auth_msg, socket) do
    {:ok, socket}
  end

  def join(topic, %{ claims: claims, resource: _resource }, socket) do
    if permitted_topic?(claims[:listen], topic) do
      { :ok, %{ message: "Joined" }, socket }
    else
      { :error, :authentication_required }
    end
  end

  def join(_room, _payload, _socket) do
    { :error, :authentication_required }
  end

  def handle_in("msg", payload, socket = %{ topic: "public:" <> _ }) do
    broadcast socket, "msg", payload
    { :noreply, socket }
  end

  def handle_in("msg", payload, socket) do
    claims = Guardian.Channel.claims(socket)
    if permitted_topic?(claims[:publish], socket.topic) do
      broadcast socket, "msg", payload
      { :noreply, socket }
    else
      { :reply, :error, socket }
    end
  end

  def permitted_topic?(nil, _), do: false
  def permitted_topic?([], _), do: false

  def permitted_topic?(permitted_topics, topic) do
    matches = fn permitted_topic ->
      pattern = String.replace(permitted_topic, ":*", ":.*")
      Regex.match?(~r/\A#{pattern}\z/, topic)
    end
    Enum.any?(permitted_topics, matches)
  end
end