postmarketOS: Experimental Sustainable Operating System
Wow, so it has been one full year since the public announcement of postmarketOS, and what a year it has been! During this time the project has grown beyond our wildest dreams; dozens of contributors making a wide variety of contributions, friendships made, bragging rights established (and lost, and re-established), Doom running on things it probably shouldn't, and we're ultimately just getting started. This post is also a celebration of all the hard work that has gone into postmarketOS in the last year, it's first year!
Before we begin with our list of changes since the last post in December and other exciting news, a quick intro for folks just now learning of this project for the first time: postmarketOS aims to be a sustainable operating system that empowers users to safely use their devices for as long as possible (ideally until they physically fall apart). The system is designed to share as many packages as possible between supported devices, with the ability to make exceptions for a few device-specific packages as appropriate/needed. This is in contrast to systems like Android, where the many device-specific changes are required to support any particular device.
At this point, postmarketOS is only meant to be used by developers. In most instances, phone calls, SMS, bluetooth or the mainline kernel won't work on your device, and there's the firmware problem.
Mainline On the Horizon
The problem is not completely resolved today, there is still a lot more work to do. But this dream on having devices with mainline kernel support is becoming more realistic now after all the amazing work done by hackers in communities like ##linux-msm and #freedreno. PostmarketOS is not the only distro with this dream, two more distributions have also decided to pursue this with (specific) phones: Pure OS and Maemo Leste.
Nexus 5: FLOSS Modem Stack With Mainline
Of course it would be nice if the device could receive phone calls and facilitate conversation. We're not quite there yet but we may be soon thanks to @Srinivas-Kandagatla, who is working on mainline audio support for the Snapdragon 820 SoC, the Linux kernel will soon be able to utilize the audio stack on Qualcomm phones like the Nexus 5. Once that is in place, we can piece the userspace functionality together.
So as of right now we can communicate with the Qualcomm Modem Interface (QMI) without using any userspace blobs. This would not have been possible without the amazing work from @scintill who packaged all necessary glue libraries provided by @andersson (#1314). The first pull request made it work with the downstream kernel of the Galaxy S4 Mini LTE followed by another PR that enabled it to work with our mainline packaging (#1381).
Thanks to: @andersson, @bshah, @flto, @MartijnBraam, @opendata26, @scintill, @Srinivas-Kandagatla
linux-postmarketosfor easy collaboration, and packaging the Qualcomm specific branch from there in a new package
linux-postmarketos-qcom. With the separate package, the patches won't break anything for non-Qualcomm devices that use an upstream kernel in postmarketOS.
Two more devices besides the Nexus 5 are also using
linux-postmarketos-qcom. The Sony Xperia Z2 (shown on the right and below, not to be confused with the tablet) ported by @opendata26 has battery capacity, temperature sensors, Wi-Fi and the display with 3D acceleration working! More great news: in theory one would need a closed source component to communicate with the cellular modem, but kernel hacker @andersson confirmed that Sony allowed him to release that as open source!
The third device is the Samsung Galaxy S5 contributed by @drebrez. It is in an early porting stage, but so far the debug serial interface, volume buttons and internal storage access are working with the
linux-postmarketos-qcom kernel. If you want to use the display with postmarketOS, you still need to use the downstream kernel (which is packaged as well). Given all of these options for kernels, we now provide you an easy way to select your preferred kernel during the installation (#1363, screenshot below).
Thanks to: @andersson, @bshah, @drebrez, @opendata26
Raspberry Pi, Samsung Galaxy Tab 10.1 and Nokia N9
Next up is the Tegra-based Samsung Galaxy Tab 10.1 with an amazing functional feature list containing Wi-Fi, 2D acceleration, VDPAU h264 hardware decoding, headphone and speaker audio as well as bluetooth! @Decatf added the device in #1279. The second photo below is a screenshot directly taken from the device.
In the third picture below you can see the Nokia N9. We introduced the mainline kernel work @filippz and @pavelmachek did in the last post. Since then the port has been updated and merged into pmbootstrap's
master branch (#1101). @pavelmachek explained why the N9 and all other devices listed in this section have their own kernel packages instead of using
linux-postmarketos-stable (like the Nokia N900): "We'll need quite a few patches for now, and it would be bad to break some other phone with patches relevant for N9. Long-term it is a goal, and we are not that far, but I don't think we should do it now."
Thanks to: @Decatf, @filippz, @drebrez, @pavelmachek, @yangxuan8282
Libhybris And Optional Proprietary Components
But don't we hate proprietary software from the deepest of our hearts? Most people in our community do. We have outlined numerous times on this blog that running proprietary blobs comes at the cost of making (security) updates almost impossible as soon as vendors drop their support. It is also much harder to verify that the binaries do not contain backdoors or other vulnerabilities.
Then why do we allow libhybris to be a part of postmarketOS? Consider the countless phones and tablets that have been produced up to this point. We can be happy if we make a few of them work with a full FLOSS stack, but for most of them this won't happen any time soon, if ever. So should free software hackers throw all of them away? For the environmental conscious, this is not an option!
This means we can't have a one size fits all solution that makes everybody happy. Some people want the full FLOSS stack, even if that means Wi-Fi and telephony do not work because they lack free firmware. Others find proprietary firmware acceptable, but do not wish to run proprietary userspace code. And then there are folks who just want to make use of all peripherals of their device, even if it requires blobs in userspace. We embrace all of these sides: nowadays you can choose how libre you want to go during the installation (#1254):
$ pmbootstrap init ... Device [qemu-amd64]: example-example This device has proprietary components, which trade some of your freedom with making more peripherals work. We would like to offer full functionality without hurting your freedom, but this is currently not possible for your device. device-example-example-nonfree-firmware: modem, Wi-Fi, accelerated GPU Enable this package? (y/n) [y]: device-example-example-nonfree-userland: accelerated GPU Enable this package? (y/n) [n]:
Thanks to: @NotKit, @ollieparanoid
Initramfs: Charging and Maximum Attention
postmarketOS is able to show a nice battery loading screen while it is "turned off" and charging, because @drebrez integrated
charging-sdl into the initramfs (#1081).
When porting pmOS to a new device, it may happen that the device doesn't do anything on first boot. The screen stays dark (or displays the OEM logo) and the USB networking does not come up. In this case we don't even know for sure if the kernel we just tested is booting at all, or if it crashed before even executing the initramfs code. But this information is crucial in order to use the right troubleshooting techniques. Thanks to @MayeulC we can install the
maximum-attention initramfs hook now, which will flash every LED it can find, as well as running the vibration motor (#1238). Be prepared to see your device wandering across the table!
Thanks to: @drebrez, @MayeulC
Just like typical Desktop Linux distributions, postmarketOS is not limited to one user interface. We have Hildon, LuneOS UI, MATE, Plasma Mobile, Weston and XFCE4 packaged already. LuneOS UI is currently disabled, because we need to re-integrate it with the latest QT upgrade in Alpine (#1459, help wanted). However, two new user interfaces have been contributed.
Since i3 is about as resource lightweight as it gets, it is a pretty good match for that classic machine and a few people are using it! The other is extending i3 with touch controls. @michitux started packaging i3touchmenu in #1343 as you can see in the photo and video of his Samsung Galaxy Note 8.0 (that pull request is not merged in pmbootstrap's
master branch yet as of writing).
Thanks to: @fxkrait, @MartijnBraam, @michitux, @sicelo
Thanks to: @duncanguthrie
#postmarketOS-lowlevel: 3 BROMs Dumped
The first was dumped via JTAG, and the second was dumped from
/dev/mem by modifying the preloader to remove some memory access restrictions that prevented Linux from reading the BROM. Together with the BROM of the MT6735P extracted by @McBitter, there's a lot of potential to understand the boot process with greater detail. After proper reverse engineering, we should better understand how boot metadata is formatted on the flash memory and possibly find new options to recover from a bad flash. In addition, a better understanding of the firmware signature verification process performed by the BROM could enable us to discover vulnerabilities and bypass these restrictions.
In order to replace components in the low level boot process on these SoCs, we also need to understand what state the BROM leaves the chip in after it has finished executing. For instance, we want to know if the BROM disables any debugging functionality, makes use of write-once registers, or disables memory regions and registers. That information might be useful for porting U-Boot SPL (Secondary Program Loader) to the platform, which would replace the proprietary preloader.
If you're asking yourself why we don't simply look at a bunch of datasheets instead, @cyrozap noted: "Basically, since we don't have much documentation on how the BROM works (and since documentation often lies or isn't the whole truth, anyways), we need to discover for ourselves how it works in order to write our own documentation for it."
Thanks to: @cyrozap, @McBitter
40 New Devices (84 Total)
We're having pretty much constant growth in the list of booting devices as you can see in the graph. Here are the new ones:
- Asus Zenfone 5
asus-t00f(shown on the right)
- Asus Eee Pad Transformer
- Geeksphone Peak
- HTC Desire 816
- Google Nexus 9
- HTC One M8
- HTC Incredible S
- InFocus New Tab F1
- Jolla phone
- LeEco Le 2
- Google Nexus 5X
- Motorola Moto G4
- Motorola Moto G (2013)
- Motorola Moto G4 Play
motorola-harpia(pictured on the bottom right)
- Motorola Droid 4
- Motorola Moto G 4G (2013)
- Motorola Moto G5 Plus
- Google Nexus 6
- Nextbit Robin
- Nokia N9
- OnePlus 2
- Raspberry Pi
- Galaxy Tab S2 9.7 WiFi (SM-T813)
- Samsung Galaxy SIII mini
- Samsung Galaxy S4 Mini
- Samsung Galaxy S5
- Samsung Galaxy S5 Mini
- Samsung Galaxy Trend
- Samsung Galaxy Tab 3 7.0
- Google Nexus 10
- Samsung Galaxy Note 8.0 (Wi-Fi)
- Samsung Galaxy Tab 10.1
- Xperia Arc
- Xperia Z3 Tablet Compact
- Sony Xperia T3
- Sony Xperia Z2
- Xiaomi Redmi 1S
- Xiaomi Redmi Note 4
- Xiaomi Redmi 4X
- ZTE Kis 3
Thanks to: everyone who ported these devices, see the contributors section in each device's wiki page.
One area that we consistently spend a lot of time improving is our swiss army knife of postmarketOS development, installation and flashing:
pmbootstrap. Let's break it down.
Continuous Integration: Build Packages And QEMU Test
Whenever someone got their new device port to a point where it boots, the porting guide recommends that they make a pull request and upstream it so everyone can benefit from the work that has been done and build upon it. In order to keep the reviewing efforts as low as possible, the following new CI checks were implemented:
#982 also has a new test case that instructs
pmbootstrap to do two full installations (one of them with XFCE4, the other one with Plasma Mobile). Both installations will then run in QEMU, and the test case will connect via SSH to the VM to verify the running processes. This has already saved us more than once from introducing bugs by accident in the installation code.
Thanks to: @ollieparanoid
Development: Local Source, Newapkbuild, X11 Menuconfig And Basic ZSH Completion
Patch development has gotten a lot easier, because now it is possible to replace the source tarball downloaded from the web on the fly with a local source code folder (#1210). Furthermore we are wrapping Alpine's
newapkbuild to quickly generate standard package build recipes for various build systems with just one shell command (#894, #1320). In addition @steamp0rt made it possible to use the GTK or QT based kernel configuration with
pmbootstrap kconfig edit [-x | -g] for those who prefer them over the ncurses based one (#1509).
As a hacker working with
pmbootstrap, you don't need to keep the growing list of things it can do in your head. Instead, there's the
--help output and the all-new README.md, which is basically a big cheat sheet with lots of example commands. You can also make use of the rudimentary ZSH autocompletion (#1232). Right now it can do basic stuff like extending
pmbootstrap newa<tab> to
pmbootstrap newapkbuild. Help is wanted (see #1517) for implementing more sophisticated functionality!
Thanks to: @ollieparanoid, @steamp0rt, @V13Axel
Installation: Custom Hostname, New Parameters --rsync and --split
@drebrez made it possible to set a custom hostname, with the device's code name being the default (#1327). Wearing down SD cards during multiple installations can be prevented with @pavelmachek's new
--rsync feature to only update what has been changed in the file system since the previous installation (#1151). Lastly a new
--split parameter allows you to split the boot and root partitions, instead of combining them to one partition with subpartitions (#1442). This makes the mainlining workflow easier, because the latter method does not work with recently mainlined devices like the Nexus 5 yet.
Thanks to: @drebrez, @pavelmachek
>350 people in the channel (+75)
- 1756 /r/postmarketOS readers (+380)
- 996 stargazers (+233)
- 867 closed PRs (+313)
- 497 closed issues (+161)
- 149 open issues (+15)
- 168 forks (+66)
- 93 watchers (+19)
- 106 contributors (
git shortlog --summary --numbered | wc -l) (+49)
Mainline Your Device!
Mainlining expert @opendata26 encourages our readers to join the effort: "One thing I think we should really highlight is that if they have decent knowledge of Linux and if they have a supported SoC - I would say S4 Pro, Snapdragon 820 (you would want UART for that but once it's running most stuff should work), 845 (same as 820, just need tons of patches on patchwork), 410, 801, 600 and probably S4 (non pro). That they can feel free to join the Matrix and that it's very likely we'll end up with a working mainline kernel." Working means in this context anything from "it boots" to having all the features working that we presented above. The latter is a bigger challenge of course.
Hackers ready for this exercise can check out the brand-new Mainlining Guide in our wiki. It provides several shortcuts to help speed you along, such as having one command that exports the right environment variables for your device and sets up a known working Alpine Linux chroot with all required packages for cross compiling the kernel without modifying your host system (#1424). The mentors listed in the guide can help you to create your device tree source file, which tells the kernel the location and parameters of all the peripherals in your little mobile computer. Because these peripherals are used over and over again in different devices, oftentimes we can just enable them without writing new drivers!
More Ways to Help Out
We have a ton of quests ready to be picked up. Whether it's simply helping out other people with thinking through how they could debug issues on their devices, fixing bugs, implementing cool new features towards the goal of having postmarketOS usable on a daily driver level or straight up hacking everything without fear: there is a lot to learn and a lot of fun to have!
- Port your phone
- Install postmarketOS on your machines if there's already a port
- Make good photos/videos of devices running postmarketOS for the wiki and /r/postmarketOS
- #1286: packaging Anbox, an experimental project that runs Android apps on regular Linux distributions. After some patching we have it compiling and displaying the "Starting..." screen (pictured on the right), but the way it configures LXC does not work out of the box for us. To move forward, one could install Anbox on Ubuntu (where it is officially supported) and add debug code to Anbox and LXC to figure out what it does differently there.
- #592: integrating nexmon so we can patch security holes in abandoned Wi-Fi firmware.
- #62 Improve or package your favorite user interface.
- Testing and/or reviewing pull requests
If your passion is writing text instead of code: we could use some help with these blog posts. It boils down to going through the closed pull requests and photos posted in the chatroom, categorizing them into logical sections before writing one paragraph after another and adding images to round it up. Contributing to any step of that process would be greatly appreciated and saves us a lot of time that we could put in other areas of the project. Get in touch with us in the postmarketOS.org issues. Another good place to exercise your writing skills to help out postmarketOS would be the wiki.
Our friends from Plasma Mobile have set up their Find Your Way page the other day. It is like a super convenient frontend to their issues tracker, which asks you a few simple questions like "Are you interested in App development?" and then takes you to the right ticket. If you could help out with any of these, you would directly contribute towards Plasma Mobile reaching 1.0 state.
No End In Sight
Here's to many more years of exciting postmarketOS developments!