HomeInstallBlogExploreTeamDonate

One Year of postmarketOS: Mainline Calling!

June 09, 201820 min. read

postmarketOS: Experimental Sustainable Operating System

Wow, so it has been one full year since the public announcement of postmarketOS, and what a year it has been! During this time the project has grown beyond our wildest dreams; dozens of contributors making a wide variety of contributions, friendships made, bragging rights established (and lost, and re-established), Doom running on things it probably shouldn't, and we're ultimately just getting started. This post is also a celebration of all the hard work that has gone into postmarketOS in the last year, it's first year!

Before we begin with our list of changes since the last post in December and other exciting news, a quick intro for folks just now learning of this project for the first time: postmarketOS aims to be a sustainable operating system that empowers users to safely use their devices for as long as possible (ideally until they physically fall apart). The system is designed to share as many packages as possible between supported devices, with the ability to make exceptions for a few device-specific packages as appropriate/needed. This is in contrast to systems like Android, where the many device-specific changes are required to support any particular device.

At this point, postmarketOS is only meant to be used by developers. In most instances, phone calls, SMS, bluetooth or the mainline kernel won't work on your device, and there's the firmware problem.

Mainline On the Horizon

In February 2017 Jeremy McNicoll presented on his progress with forward porting the Google Nexus 5X/6P. He explains the technical difficulties of replacing the downstream Linux kernel that you find on your device after you bought it with the latest upstream version. Towards the end he mentioned on his "lessons learned" slide: "Maybe 10 years ago I would have said we can cure world hunger, we can get the whole thing working." Then he shook his head. "I'm more realistic now."

This is how it felt one and a half years ago. It was hard to imagine that we could have phones that would run the upstream kernel. We did not have a feasible method of patching all of the critical security holes in the core of the operating system once the vendor's ridiculously short support cycle ends.

The problem is not completely resolved today, there is still a lot more work to do. But this dream on having devices with mainline kernel support is becoming more realistic now after all the amazing work done by hackers in communities like ##linux-msm and #freedreno. PostmarketOS is not the only distro with this dream, two more distributions have also decided to pursue this with (specific) phones: Pure OS and Maemo Leste.

Even established vendors seem be coming around to the idea. In March of this year, the Linaro Connect Opening Keynote spent 15 minutes pleading with their big list of member companies that upstream kernel support is the way to go: "Very few embedded SoCs are supported properly upstream. That is changing, it has to change. [...] and most importantly, we have to do that upstream. Because nothing else scales. Testing just your own version doesn't actually help everyone else. And as a result you don't get the benefit of everybody else's testing."

Nexus 5: FLOSS Modem Stack With Mainline

You might remember @bshah's photo of the Nexus 5 running mainline with a flipped and distorted screen from December. @flto continued his work: the display works without problems now. But it gets even better: the touch screen is working, 3D acceleration is enabled with the open source freedreno userspace driver, Wi-Fi works, and the best part is that @MartijnBraam was able to send SMS and initialize a call via command line as well as getting the connectivity signal from the modem through oFono displayed in Plasma Mobile (#1502). All of that without proprietary userspace blobs!

Of course it would be nice if the device could receive phone calls and facilitate conversation. We're not quite there yet but we may be soon thanks to @Srinivas-Kandagatla, who is working on mainline audio support for the Snapdragon 820 SoC, the Linux kernel will soon be able to utilize the audio stack on Qualcomm phones like the Nexus 5. Once that is in place, we can piece the userspace functionality together.

So as of right now we can communicate with the Qualcomm Modem Interface (QMI) without using any userspace blobs. This would not have been possible without the amazing work from @scintill who packaged all necessary glue libraries provided by @andersson (#1314). The first pull request made it work with the downstream kernel of the Galaxy S4 Mini LTE followed by another PR that enabled it to work with our mainline packaging (#1381).

Thanks to: @andersson, @bshah, @flto, @MartijnBraam, @opendata26, @scintill, @Srinivas-Kandagatla

linux-postmarketos-qcom

As we are running bleeding edge technology here, we need to use kernel patches that have not made their way into the mainline kernel yet. These patches are tiny in size, and the patch authors are trying to upstream them (in contrary to kernels from Android with vendor patches on the order of a hundred thousand to one million lines of changes, as the Linaro keynote mentions). We decided to set up a staging repository linux-postmarketos for easy collaboration, and packaging the Qualcomm specific branch from there in a new package linux-postmarketos-qcom. With the separate package, the patches won't break anything for non-Qualcomm devices that use an upstream kernel in postmarketOS.

Two more devices besides the Nexus 5 are also using linux-postmarketos-qcom. The Sony Xperia Z2 (shown on the right and below, not to be confused with the tablet) ported by @opendata26 has battery capacity, temperature sensors, Wi-Fi and the display with 3D acceleration working! More great news: in theory one would need a closed source component to communicate with the cellular modem, but kernel hacker @andersson confirmed that Sony allowed him to release that as open source!

The third device is the Samsung Galaxy S5 contributed by @drebrez. It is in an early porting stage, but so far the debug serial interface, volume buttons and internal storage access are working with the linux-postmarketos-qcom kernel. If you want to use the display with postmarketOS, you still need to use the downstream kernel (which is packaged as well). Given all of these options for kernels, we now provide you an easy way to select your preferred kernel during the installation (#1363, screenshot below).

Thanks to: @andersson, @bshah, @drebrez, @opendata26

Raspberry Pi, Samsung Galaxy Tab 10.1 and Nokia N9

Besides Qualcomm devices, we have a hand full of additional new ports running kernels that are very close to mainline. @drebrez tweaked postmarketOS to run on all Raspberry Pi models up to version 3, building on the existing support in Alpine (#1501). @yangxuan8282 did a lot of testing and took this photo of XFCE4 running on a small touch screen directly on the Pi. He also has multiple pull requests to support the Raspberry Pi 3B+.

Next up is the Tegra-based Samsung Galaxy Tab 10.1 with an amazing functional feature list containing Wi-Fi, 2D acceleration, VDPAU h264 hardware decoding, headphone and speaker audio as well as bluetooth! @Decatf added the device in #1279. The second photo below is a screenshot directly taken from the device.

In the third picture below you can see the Nokia N9. We introduced the mainline kernel work @filippz and @pavelmachek did in the last post. Since then the port has been updated and merged into pmbootstrap's master branch (#1101). @pavelmachek explained why the N9 and all other devices listed in this section have their own kernel packages instead of using linux-postmarketos-stable (like the Nokia N900): "We'll need quite a few patches for now, and it would be bad to break some other phone with patches relevant for N9. Long-term it is a goal, and we are not that far, but I don't think we should do it now."

Thanks to: @Decatf, @filippz, @drebrez, @pavelmachek, @yangxuan8282

Libhybris And Optional Proprietary Components

@NotKit's work on packaging libhybris has been merged (#1402), which is the base for making proprietary userspace drivers from Android work with non-Android Linux distributions. More streamlining is necessary to actually use this when installing postmarketOS. However he made several demos already, including a port of postmarketOS with libhybris and 3D acceleration running on the Gemini PDA (photo from here).

But don't we hate proprietary software from the deepest of our hearts? Most people in our community do. We have outlined numerous times on this blog that running proprietary blobs comes at the cost of making (security) updates almost impossible as soon as vendors drop their support. It is also much harder to verify that the binaries do not contain backdoors or other vulnerabilities.

Then why do we allow libhybris to be a part of postmarketOS? Consider the countless phones and tablets that have been produced up to this point. We can be happy if we make a few of them work with a full FLOSS stack, but for most of them this won't happen any time soon, if ever. So should free software hackers throw all of them away? For the environmental conscious, this is not an option!

This means we can't have a one size fits all solution that makes everybody happy. Some people want the full FLOSS stack, even if that means Wi-Fi and telephony do not work because they lack free firmware. Others find proprietary firmware acceptable, but do not wish to run proprietary userspace code. And then there are folks who just want to make use of all peripherals of their device, even if it requires blobs in userspace. We embrace all of these sides: nowadays you can choose how libre you want to go during the installation (#1254):

$ pmbootstrap init
...
Device [qemu-amd64]: example-example
This device has proprietary components, which trade some of your freedom with making more
peripherals work. We would like to offer full functionality without hurting your freedom,
but this is currently not possible for your device.
device-example-example-nonfree-firmware: modem, Wi-Fi, accelerated GPU
Enable this package? (y/n) [y]:
device-example-example-nonfree-userland: accelerated GPU
Enable this package? (y/n) [n]:

Thanks to: @NotKit, @ollieparanoid

Initramfs: Charging and Maximum Attention

postmarketOS is able to show a nice battery loading screen while it is "turned off" and charging, because @drebrez integrated charging-sdl into the initramfs (#1081).

When porting pmOS to a new device, it may happen that the device doesn't do anything on first boot. The screen stays dark (or displays the OEM logo) and the USB networking does not come up. In this case we don't even know for sure if the kernel we just tested is booting at all, or if it crashed before even executing the initramfs code. But this information is crucial in order to use the right troubleshooting techniques. Thanks to @MayeulC we can install the maximum-attention initramfs hook now, which will flash every LED it can find, as well as running the vibration motor (#1238). Be prepared to see your device wandering across the table!

Thanks to: @drebrez, @MayeulC

New UIs

Just like typical Desktop Linux distributions, postmarketOS is not limited to one user interface. We have Hildon, LuneOS UI, MATE, Plasma Mobile, Weston and XFCE4 packaged already. LuneOS UI is currently disabled, because we need to re-integrate it with the latest QT upgrade in Alpine (#1459, help wanted). However, two new user interfaces have been contributed.

i3wm

Tiling window managers like i3 make as much use of the screen space as possible. If you have one window open, it is always full screen, two windows split the screen in half and so on. When thinking about this, that is not so different from what today's mainstream smartphone user interfaces do. The similarities end with the fact that you can use the touch screen to manage application windows on the phone, while in a tiling WM you would use the keyboard.

We have found two ways to use i3 on a mobile device. @MartijnBraam did the obvious one with a keyboard in #1225, which works well with the Nokia N900 (photo on the right).

Since i3 is about as resource lightweight as it gets, it is a pretty good match for that classic machine and a few people are using it! The other is extending i3 with touch controls. @michitux started packaging i3touchmenu in #1343 as you can see in the photo and video of his Samsung Galaxy Note 8.0 (that pull request is not merged in pmbootstrap's master branch yet as of writing).

Thanks to: @fxkrait, @MartijnBraam, @michitux, @sicelo

Matchbox

@duncanguthrie had fun with porting the old school Matchbox window manager to postmarketOS (#1420). This is part of his quest of bringing back a classic PDA-like experience, as detailed in the Handheld wiki article he wrote. The photo on the right shows it running on the Samsung Galaxy Tab 3 7.0.

Thanks to: @duncanguthrie

#postmarketOS-lowlevel: 3 BROMs Dumped

Speaking of free firmware: our recently established #postmarketOS-lowlevel crew is taking baby steps towards liberating bootloaders for MediaTek devices. Around the time of the announcement blog post in April, quite a few people have joined the channel and started contributing. Notably @cyrozap has managed to dump the Boot ROMs (BROMs) of the MT6797 (Helio X20) and the MT6737M.

The first was dumped via JTAG, and the second was dumped from /dev/mem by modifying the preloader to remove some memory access restrictions that prevented Linux from reading the BROM. Together with the BROM of the MT6735P extracted by @McBitter, there's a lot of potential to understand the boot process with greater detail. After proper reverse engineering, we should better understand how boot metadata is formatted on the flash memory and possibly find new options to recover from a bad flash. In addition, a better understanding of the firmware signature verification process performed by the BROM could enable us to discover vulnerabilities and bypass these restrictions.

In order to replace components in the low level boot process on these SoCs, we also need to understand what state the BROM leaves the chip in after it has finished executing. For instance, we want to know if the BROM disables any debugging functionality, makes use of write-once registers, or disables memory regions and registers. That information might be useful for porting U-Boot SPL (Secondary Program Loader) to the platform, which would replace the proprietary preloader.

If you're asking yourself why we don't simply look at a bunch of datasheets instead, @cyrozap noted: "Basically, since we don't have much documentation on how the BROM works (and since documentation often lies or isn't the whole truth, anyways), we need to discover for ourselves how it works in order to write our own documentation for it."

Thanks to: @cyrozap, @McBitter

40 New Devices (84 Total)

Graph of the number of devices in the repository over time

We're having pretty much constant growth in the list of booting devices as you can see in the graph. Here are the new ones:

Thanks to: everyone who ported these devices, see the contributors section in each device's wiki page.

pmbootstrap Improvements

One area that we consistently spend a lot of time improving is our swiss army knife of postmarketOS development, installation and flashing: pmbootstrap. Let's break it down.

Continuous Integration: Build Packages And QEMU Test

Whenever someone got their new device port to a point where it boots, the porting guide recommends that they make a pull request and upstream it so everyone can benefit from the work that has been done and build upon it. In order to keep the reviewing efforts as low as possible, the following new CI checks were implemented:

#982 also has a new test case that instructs pmbootstrap to do two full installations (one of them with XFCE4, the other one with Plasma Mobile). Both installations will then run in QEMU, and the test case will connect via SSH to the VM to verify the running processes. This has already saved us more than once from introducing bugs by accident in the installation code.

Thanks to: @ollieparanoid

Development: Local Source, Newapkbuild, X11 Menuconfig And Basic ZSH Completion

Patch development has gotten a lot easier, because now it is possible to replace the source tarball downloaded from the web on the fly with a local source code folder (#1210). Furthermore we are wrapping Alpine's newapkbuild to quickly generate standard package build recipes for various build systems with just one shell command (#894, #1320). In addition @steamp0rt made it possible to use the GTK or QT based kernel configuration with pmbootstrap kconfig edit [-x | -g] for those who prefer them over the ncurses based one (#1509).

As a hacker working with pmbootstrap, you don't need to keep the growing list of things it can do in your head. Instead, there's the --help output and the all-new README.md, which is basically a big cheat sheet with lots of example commands. You can also make use of the rudimentary ZSH autocompletion (#1232). Right now it can do basic stuff like extending pmbootstrap newa<tab> to pmbootstrap newapkbuild. Help is wanted (see #1517) for implementing more sophisticated functionality!

Thanks to: @ollieparanoid, @steamp0rt, @V13Axel

Installation: Custom Hostname, New Parameters --rsync and --split

@drebrez made it possible to set a custom hostname, with the device's code name being the default (#1327). Wearing down SD cards during multiple installations can be prevented with @pavelmachek's new --rsync feature to only update what has been changed in the file system since the previous installation (#1151). Lastly a new --split parameter allows you to split the boot and root partitions, instead of combining them to one partition with subpartitions (#1442). This makes the mainlining workflow easier, because the latter method does not work with recently mainlined devices like the Nexus 5 yet.

Thanks to: @drebrez, @pavelmachek

Raw Numbers

Get Involved

Mainline Your Device!

Mainlining expert @opendata26 encourages our readers to join the effort: "One thing I think we should really highlight is that if they have decent knowledge of Linux and if they have a supported SoC - I would say S4 Pro, Snapdragon 820 (you would want UART for that but once it's running most stuff should work), 845 (same as 820, just need tons of patches on patchwork), 410, 801, 600 and probably S4 (non pro). That they can feel free to join the Matrix and that it's very likely we'll end up with a working mainline kernel." Working means in this context anything from "it boots" to having all the features working that we presented above. The latter is a bigger challenge of course.

Hackers ready for this exercise can check out the brand-new Mainlining Guide in our wiki. It provides several shortcuts to help speed you along, such as having one command that exports the right environment variables for your device and sets up a known working Alpine Linux chroot with all required packages for cross compiling the kernel without modifying your host system (#1424). The mentors listed in the guide can help you to create your device tree source file, which tells the kernel the location and parameters of all the peripherals in your little mobile computer. Because these peripherals are used over and over again in different devices, oftentimes we can just enable them without writing new drivers!

More Ways to Help Out

We have a ton of quests ready to be picked up. Whether it's simply helping out other people with thinking through how they could debug issues on their devices, fixing bugs, implementing cool new features towards the goal of having postmarketOS usable on a daily driver level or straight up hacking everything without fear: there is a lot to learn and a lot of fun to have!

Specific examples:

  • Port your phone
  • Install postmarketOS on your machines if there's already a port
  • Make good photos/videos of devices running pmOS for the wiki and /r/postmarketOS the Fediverse
  • #1286: packaging Anbox, an experimental project that runs Android apps on regular Linux distributions. After some patching we have it compiling and displaying the "Starting..." screen (pictured on the right), but the way it configures LXC does not work out of the box for us. To move forward, one could install Anbox on Ubuntu (where it is officially supported) and add debug code to Anbox and LXC to figure out what it does differently there.
  • #592: integrating nexmon so we can patch security holes in abandoned Wi-Fi firmware.
  • #62 Improve or package your favorite user interface.
  • Testing and/or reviewing pull requests

If your passion is writing text instead of code: we could use some help with these blog posts. It boils down to going through the closed pull requests and photos posted in the chatroom, categorizing them into logical sections before writing one paragraph after another and adding images to round it up. Contributing to any step of that process would be greatly appreciated and saves us a lot of time that we could put in other areas of the project. Get in touch with us in the postmarketOS.org issues. Another good place to exercise your writing skills to help out postmarketOS would be the wiki.

Our friends from Plasma Mobile have set up their Find Your Way page the other day. It is like a super convenient frontend to their issues tracker, which asks you a few simple questions like "Are you interested in App development?" and then takes you to the right ticket. If you could help out with any of these, you would directly contribute towards Plasma Mobile reaching 1.0 state.

No End In Sight

While this blog post is being being written, the next stunning pull requests are already rolling in (e.g. Nintendo Switch in #1534). Hackers are helping out each other in the various #postmarketOS chats and coming up with cool new ideas of using postmarketOS or simply bringing it closer to something acceptable for daily use. Some people who just joined the project have already made critical bug fixes like @michitux in alpinelinux/aports#3889. Others silently expand our wiki. We get stuff done and have a ton of fun while doing it, and everyone involved should be proud of what we've accomplished together in just one year! Whoever has the chance to visit Akademy 2018 in Vienna, make sure to meet the few of us who will celebrate there and watch @ata2001's presentation about postmarketOS live on stage!

Here's to many more years of exciting postmarketOS developments!

What do you think?