Over the last couple of years, Pidgin, and by extension our non-profit corporation called Instant Messaging Freedom, Inc., has received sponsorship from a hosting company called DigitalOcean. DigitalOcean provides a variety of hosting services, including virtual private servers, managed kubernetes clusters, and so on. They also provide sponsorship for open source projects, whereby they provide credits to use to offset the costs of their services.
This sponsorship has been the source of Pidgin and Instant Messaging Freedom’s primary infrastructure. Currently we run this website, our JetBrains Hub centralized authentication system (running at hub.imfreedom.org), our JetBrains YouTrack instance (running at issues.imfreedom.org), our Mercurial hosting solution, HGKeeper, that runs at keep.imfreedom.org, and a number of other tools and services all from the DigitalOcean-provided infrastructure. In the coming weeks and months, we will have all of our infrastructure, including e-mail and our previous bug tracking/wiki system, running entirely in DigitalOcean’s datacenters thanks to their generous sponsorship.
We get a lot of people asking why we self-host when there are resources like the ubiquitous GitHub that would do a lot of the work for us. That’s a fair question, and admittedly we’re not always the most patient when answering. The answer is a bit longer than you’d initially expect.
First and foremost, Pidgin is an end-user application. Sure, we have lots of developers who use Pidgin, but our target audience is end users, not developers. End users who aren’t developers are going to find the user experience of a tool like GitHub, GitLab, or similar to be very lacking. In the current landscape, self-hosting is essentially the only way to get the end-user-facing aspects of our infrastructure to be as friendly as possible while still being relatively easy to maintain.
For many years, we used SourceForge for our hosting needs. This included our ancient PHP-based website, our CVS source code hosting, issue tracking, mailing lists, and our downloads. SourceForge was, and still is, a gracious host that served us very well. Around the time we changed our name, we secured alternative hosting for much of those items because the SourceForge issue trackers were cumbersome and unwieldy and we wanted to move away from CVS and Subversion and toward distributed version control.
We migrated to Edgewall Trac for issue tracking and wiki management, and Monotone for version control. At the same time, since SourceForge couldn’t handle these tools, we migrated to a hosting provider that our friends at the Adium project were using. This provider donated hosting to Adium and agreed to donate hosting to us as well—in our case, two virtual private servers. That hosting provider exited the hosting business in 2020, after both Adium and Pidgin had been with them for well in excess of 10 years.
We also were using a binary hosting provider to host Continuous Integration artifacts and our releases as an alternative to SourceForge for those who still held animosity and distrust for SourceForge for actions under previous ownership and management. This hosting service has announced it is exiting from the hosting business in mid 2021.
However, the biggest blow to us was Atlassian BitBucket dropping Mercurial support. We had migrated to Mercurial after our time with Monotone proved that we needed a different version control tool. After that migration, in our development workflow for Pidgin 3.0.0, we had become dependent on the pull request workflow, issue tracking tools, and continuous integration system provided to us there. Fortunately, Atlassian had the courtesy to announce the removal of Mercurial support far enough in advance that Gary was able to write HGKeeper and get our repositories migrated away from BitBucket in time to prevent the loss of our repositories. (We have no desire to migrate to git and GitHub, GitLab, or similar, for reasons beyond the scope of this post.)
Most recently, just within the last few days a tool we were using to monitor our services, UptimeRobot, announced and then implemented severe curtailments to the functionality of their free monitoring offering. We were using UptimeRobot to monitor and alert on a variety of our infrastructure’s components and services; the changes to the free offering makes it no longer viable for us, thus we were forced to migrate to another tool.
All of these losses have made us rather wary of becoming too dependent on specialized hosting providers and services. We’re now much more inclined to build our own infrastructure in a generic, repeatable way that allows us to migrate to a new generalized hosting provider if we ever need to.
Quite honestly, two reasons. First, they were the only hosting provider we knew of at the time which offered managed kubernetes clustering. Second, the Open Source project sponsorship. Gary, in particular, wanted the managed kubernetes functionality due to his previous experience with it. We became aware of the sponsorship later and it was essentially a bonus to us.
Without getting too technical, we have five “Droplet” virtual private servers, four of which are a managed kubernetes cluster. The fifth will host our e-mail and mailing lists when we are able to sort out a few challenges. We also have a “small” load balancer (DigitalOcean’s term) to handle ingress into the various services running on the cluster, including spreading across multiple instances of a given service within the cluster.
Our aim is to run everything possible in the cluster, with all the configuration (except for secrets) well-defined and version-controlled. This allows us to have automatic service recovery in the event of a problem or a need to perform maintenance on one of the Droplets (such as upgrading to new kubernetes releases or container builds). Where possible, we also aim to run at least two instances of services within the cluster to provide a measure of high availability. For example, the container which serves this website runs on at least two nodes in the cluster at any given time and the load balancer will spread the traffic across all running instances of the container. If an instance has a failure, the load balancer stops sending traffic to that instance until the cluster recycles the container and resolves the problem. This is all fully automatic, with no need for human intervention. The cluster self-heals for the vast majority of container failures, which was a huge part of the appeal to us, especially Gary, due to limited time to deal with administration.
We’ve even moved our Trac instance, now read-only except to a select few long-time Pidgin developers, into the cluster to reduce the overall complexity of our hosting and allow another hosting provider to retire the aging dedicated physical server currently running Trac.
With the functionality built into the managed kubernetes clustering, TLS certificates, whether for HTTPS (web services), XMPP messaging, or any other TLS-capable service, are automatically managed. Using the Let’s Encrypt certificate authority and the integration into the clustering, all our TLS certificates are automatically issued, renewed, and replaced dynamically as needed with no intervention required.
Within the cluster we also run the website and the protocol documentation wiki for Instant Messaging Freedom, Inc. As mentioned before, Hub, YouTrack, and HGKeeper also run within the cluster, all in their own containers, along with some other tools that we aren’t yet able to make public.
Finally, we also run the Prosody IM XMPP server in the cluster. This service provides Pidgin and Instant Messaging Freedom, Inc. with instant messaging services over the XMPP protocol. It also provides the basis for the PidginChat service many of our users now enjoy.
In conclusion, the Pidgin project and Instant Messaging Freedom, Inc. would like to thank DigitalOcean for their generous sponsorship that allows us to continue to develop free and open source messaging software for the benefit of the entire world!