lxgr's blog//blog.lxgr.net/2015-04-30T13:32:00+02:00Booting multiple Ubuntu versions with EFI2015-04-30T13:32:00+02:00lxgrtag:blog.lxgr.net,2015-04-30:posts/2015/04/30/grub-efi-multiboot/<p>For an upcoming project, I will have to use Ubuntu 14.04, and since I didn't want to downgrade my main Ubuntu install on my laptop, I decided to install the second version on a spare partition of my primary harddisk. Sounds easy, right? That's what I thought too, and I was very wrong.</p>
<p>I expected to be able to simply point the Ubuntu installer to the spare partition and wait for the automatic setup to complete, like I used to when I was using plain old BIOS and MBRs to boot my system. My current laptop, however, supports something called UEFI and Secure Boot, and since the "secure" part piqued my interest, I had decided to give it a try. I have been using this setup for dual-booting Ubuntu and Windows 8 for more than a year without any problems now.</p>
<p>After watching the installer copy all the necessary files of Ubuntu 14.04 to the disk and installing the bootloader, I booted into the new system, and everything worked as expected so far – I saw the new, old 14.04 desktop and was even able to open my main, encrypted 15.04 partition using Nautilus. But when I wanted to boot back into my main system, I realized that something had gone wrong: My 15.04 install didn't show up anywhere in the boot process. Not in the EFI boot menu of my laptop I can access by pressing F12 during the boot process (I used that to boot the Windows, because somehow the secure boot stuff interfered with grub's chain loading), and also not in grub itself – it looked as if the installation had never existed in the first place (except for the fact that I was able to see all of its file in 14.04, of course).</p>
<p>After the initial rush of panic had subsided, I was able to restore the system to its original state by the "usual" process of <a href="http://howtoubuntu.org/how-to-repair-restore-reinstall-grub-2-with-a-ubuntu-live-cd">mounting and chrooting</a> my previous system and executing <code>grub-install</code>. This worked, but now I was not able to boot into the new system anymore, besides executing all those steps again from within the old system. I realized I had to dig into the details of the Linux boot process on EFI (and secure boot) if I wanted to accomplish the triple-boot setup I had in mind. If you are unfamiliar with UEFI, I would recommend you to <a href="https://www.happyassassin.net/2014/01/25/uefi-boot-how-does-that-actually-work-then/">read an introduction</a> before reading the following paragraphs.</p>
<p>The main problem seemed to be that the <code>grub-install</code> utility of both Ubuntu systems (which is also executed from the installer) wrote to the same location – I just didn't know what that location was. I started to dig into the details of the EFI boot process – from previous triple-boot experiments on a Mac I vaguely remembered something about a so-called "EFI system partition" where the initial boot loaders of all installed operating systems are stored, and also that there were some parts of the UEFI configuration that were stored in a non-volatile memory on the mainboard. This is very different from the "legacy" boot process, where the trusty BIOS simply loads a chunk of code stored in the MBR of some disk (which can be configured in the BIOS setup, but usually defaults to a sequence of CD-ROM/USB/primary HDD) and executes it. That process is both simple (it doesn't depend on any stable configuration storage within the PC) and robust (in my experience, it was possible to migrate to a new machine simply by swapping the hard drive!), but is also showing its age – things like a dual-boot setup of uncooperating operating systems quickly become a mess, as everybody who has ever installed Windows after Linux on the same machine probably knows. Additionally, there is no mechanism that allows checking the integrity of the system before it is booted, enabling malware that hooks itself into the boot process and is virtually undetectable by any software or operating system mechanism.</p>
<p>UEFI solves the problem of multiple operating systems by specifying the "EFI system partition", which is basically just a plain old FAT partition with a special partition flag and a standardized folder structure where every operating system on the disk or even machine (more on that later) can store its initial bootloader as an executable file. This is a much more robust and future-proof way of storing the first-stage bootloader than the very limited MBR scheme that basically only allows a single primary bootloader which has to locate and execute all secondary boot loaders of all operating systems on the drive. However, it is unfortunately not enough (at least on my computer!) to just dump a bootloader in the correct location (which would have been a <em>really</em> nice EFI feature!) – the corresponding operating system also has to tell the EFI about the new bootloader (both the EFI disk's UUID it resides on and the path on that disk), which then stores that information in its non-volatile configuration memory (a.k.a. NVRAM).</p>
<p>To sum the EFI boot process up, you need a folder on the EFI system partition containing your operating system's boot loader as an EFI binary (which in turn might be the first stage of a multi-stage boot loader that simply locates and executes its remaining parts) and a pointer (i.e. disk and path) to that file in the NVRAM. In the case of secure boot, the EFI binary will also be signed by some "trusted" entity, which could be your operating system vendor or amusingly Microsoft (even though you aren't actually using their operating system – this is because many hardware vendors opted to include their keys in their firmware, which was cause for much political discussion when secure boot was initially introduced).</p>
<p>Fortunately, <code>grub-install</code> takes care about all of that automatically as long as all the correct flags are supplied to it – but unfortunately, in the case of Ubuntu and secure boot, this only works for a single installation per <em>machine</em> (i.e. <em>not</em> per disk, which kept me puzzling for hours!). I'm no expert on secure boot, but I think that this might not be easily fixable by Ubuntu, depending on how exactly the signature mechanism is implemented.</p>
<p>When invoked with no parameters, Ubuntu's <code>grub-install</code>, on an EFI system, installs its signed bootloader to the EFI system partition that is mounted in <code>/boot/efi</code> (which might or might not be the one you want to use on a multi-disk setup!), in the folder <code>/EFI/ubuntu/</code>. The signed loader consists of a shim signed by Microsoft that subsequently executes the actual EFI loader called <code>grubx64.efi</code> in the same directory. Finally, there is a <code>grub.cfg</code> configuration file which contains a pointer (both as a disk UUID and as a GPT number) to the "regular" grub boot disk, which is usually your Linux root partition (or a separate boot partition if you are using an encrypted root device).</p>
<p>Initially, the problem only seemed to be that both Ubuntu installs were trying to install their bootloader and configuration to the same EFI subdirectory – I thought that if I were to somehow convince <code>grub-install</code> to install the EFI loader to some other subdirectory of <code>/EFI</code>, I would be able to select the Ubuntu I wanted from the EFI boot screen. <code>grub-install</code> conveniently has an option for exactly that; you can either change the value of the <code>GRUB_DISTRIBUTOR</code> variable in <code>/etc/default/grub</code>, or directly supply it using the <code>--bootloader-id</code> parameter. When invoking <code>grub-install</code> this way, you can see that a new folder in <code>/EFI</code> will be created using the supplied name (and registered with the EFI NVRAM). Unfortunately, while I was able to boot from the newly created boot entry, I didn't seem to be able to change the disk that grub was booting from in any way. It took me hours to find out why.</p>
<p>Remember that grub uses the value stored in <code>/EFI/<loadername>/grub.cfg</code> to determine the disk where it will continue loading. With a lot of trial-and-error experimentation, I was finally able to determine that regardless of which boot entry I was using in the EFI boot manager, grub would always read the same grub.cfg from <code>/EFI/ubuntu</code>, <strong>regardless of the actual bootloader location</strong> (i.e. subfolder of <code>/EFI</code>)! This location is actually hardcoded in the <code>grubx64.efi</code> binary, which can be verified by using <code>strings</code> or simply opening it with a hex editor. This means that regardless of the Ubuntu install from which <code>grub-install</code> was executed, only the system that <em>most recently</em> installed the loader in the <em>default location</em> <code>/EFI/ubuntu</code> was actually able to change the partition that grub would boot from. (I think I found out about that hard-coded string from some bug report, which I will try to find and reference here.)</p>
<p>If the hard-coded string is modified to reflect the actual location of the boot entry in the <code>/EFI</code> directory everything works as expected (with secure boot enforcing disabled)! Now why would the Ubuntu team be so stupid to hard-code a string that obviously would better be supplied by a parameter? The answer is secure boot: If you enable signature enforcing in the EFI configuration, the modified bootloader stops working. It seems that the string within the binary is covered in the asymmetric signature Canonical uses to certify their bootloader; they could either modify it (and break all systems where secure boot is enforced) or leave it as it is (and break multi-booting). They seem to have decided on the latter. (Maybe there is also a third way, where the configuration file location could be encoded relative to the binary location, i.e. <code>./grub.cfg</code>, but I don't know enough about EFI to say whether such a thing is possible.)</p>
<p>As I later realized, there is an easier way than modifying the signed grub binary. Since secure boot doesn't work with the modified loader anyway, I tried to invoke <code>grub-install</code> with the <code>--no-uefi-secure-boot</code> parameter and examined the resulting bootloader: Without secure boot, there is only a single EFI executable that is also called <code>grubx64.efi</code> (which confused me to no end, since the other files are <em>not</em> cleaned up by <code>grub-install</code>, and I assumed that the configuration file was still working), but that has a much simpler internal structure and importantly has the boot disk location hardcoded. This wasn't as easy to find as the suspicious <code>/EFI/ubuntu</code> string – it is only some kind of relative disk ID like <code>(,gpt2)</code>, if your boot partition is the second partition of the volume on which the EFI partition resides, but a complete disk UUID if the boot partition is located on a <em>different</em> disk.</p>
<p>Finally, here is the complete guide on how to install two Ubuntu versions on a single disk:</p>
<ul>
<li>Disable secure boot in your EFI settings.</li>
<li>Install the first Ubuntu system on the disk. (If it already exists and you have spare space on your disk, you can obviously skip this.)</li>
<li>
<p>Backup the boot entry of the first disk by reinstalling it from within the system using a different name, without secure boot:</p>
<div class="highlight"><pre>grub-install --bootloader-id=myfirstubuntu --no-uefi-secure-boot
</pre></div>
</li>
<li>
<p>Install the second system using the regular Ubuntu installer where you want it. This will break the boot entry of the first system called <code>ubuntu</code>, but not the backup you just created.</p>
</li>
<li>Boot the second system and create a backup of the bootloader, e.g. <code>grub-install --bootloader-id=mysecondubuntu --no-uefi-secure-boot</code></li>
<li>(only if you want to primarily boot the first system) Boot the first system using your computer's EFI boot menu and execute <code>grub-install</code> without any parameters.</li>
</ul>
<p>Congratulations, you now have <em>two</em> Ubuntus running on a single machine!</p>
<p>If you want to use a similar setup, but using more than one disk, you can basically use the same steps if you don't mind that the same EFI partition of the first disk will be used for both systems, which means that you can never format or remove that disk without also disrupting the system on the other disk. grub will just put a pointer to the second disk in its binary that is executed from the EFI partition on the first disk, which should theoretically even survive partition and disk renumbering (but don't count on it!).</p>
<p>If that is a problem for you, there is also the possibility of using a second EFI partition on the second disk, but the Ubuntu installer will make your life even harder by stubbornly insisting to use the EFI partition on the first disk; I was able to solve this only by creating a backup of the first system's bootloader as described above, installing the second system, mounting the second EFI partition in <code>/boot/efi</code> instead of the first one and rerunning <code>grub-install --bootloader-id=...</code>.</p>
<p>You can verify if everything has been setup as you want it by examining the EFI directory on the EFI system partition(s) on your disk(s) as well as the output of <code>efibootmgr -v</code>, which lists the content of the boot list in the EFI NVRAM.</p>
<p>There is also an option <code>--removable</code> which supposedly sets up the EFI directory on a removable device, which looks a bit different than for internal devices and importantly doesn't create an NVRAM entry (which wouldn't be available on different machines anyway). You might be able to use that to boot from an internal disk too, but I have not tried that approach, however.</p>
<p>Of course, if that sounds like a lot of headache and your computer still supports the legacy BIOS boot process (a.k.a CSM in EFI parlance), you can just install the second system on a different disk with an MBR bootloader and configure your EFI for both CSM and EFI booting if it supports that; then you can just select the EFI entry of the first system or the second disk (which will start the second system's boot loader) in the EFI boot menu.</p>
<p>Let me know if you actually made it through that big wall of text and were able to solve your EFI boot problems in the comments!</p>
<h1>Update (2015-12-11):</h1>
<p>There is an <a href="https://www.kubuntuforums.net/archive/index.php/t-68588.html">interesting discussion</a> about the whole topic in the Kubuntu forums. Apparently, it should also be possible to use multiple EFI partitions to get multiple instances of Ubuntu working with secure boot. Thanks for that idea, and sorry for being unreachable. I will probably have to add my mail address here sometime. In the meantime, you can try "me" at lxgr dot net.</p>How to fix video tearing on Chrome/Chromium and Compiz2015-04-27T18:27:00+02:00lxgrtag:blog.lxgr.net,2015-04-27:posts/2015/04/27/chrome-linux-vsync/<p>One thing I really like about Netflix is their excellent device and browser support. Unlike a certain other streaming service (the one from the company also selling books and clouds), which wouldn't allow watching their streams using an Android tablet (bizarrely, smartphones were somehow allowed...?) and requires Flash and/or Silverlight in the browser, Netflix only requires a browser that supports the <a href="https://en.wikipedia.org/wiki/Media_Source_Extensions">HTML5 Media Source Extensions</a> (plain "old" HTML5 video tag support is not enough), the <a href="http://www.w3.org/TR/encrypted-media/">HTML5 DRM extensions</a> a.k.a. EME and one of the three supported DRM plugins (Microsoft's Playready, Apple's Fairplay, or Google's Widevine; used by and shipping with Windows/IE, Safari on OS X and Chrome, respectively). Of course, the DRM requirement is annoying (somehow, these things tend to be broken sooner rather than later and only make things inconvenient for legitimate customers), but it is much better than those horribly outdated and inefficient browser plugins.</p>
<p>The only thing that was annoying me was a very annoying case of <a href="https://en.wikipedia.org/wiki/Screen_tearing">screen tearing</a> using Chrome in fullscreen on my laptop running Unity/Ubuntu. This was never an issue for me with other browsers (e.g. Firefox), video players or games, so I initially suspected a bug in Chromium and <a href="http://code.google.com/p/chromium/issues/detail?id=344141">filed a bug report</a>. Thanks to some <a href="https://code.google.com/p/chromium/issues/detail?id=344141#c20">very helpful comments</a> on that bug's discussion thread, I have been able to finally understand what is going on.</p>
<p>However, as it turns out, the problem seems to be actually caused by Compiz (the default window manager of Unity on Ubuntu), or more specifically, its "unredirect output" feature for fullscreen applications. Compiz is a <a href="https://en.wikipedia.org/wiki/Compositing_window_manager">compositing window manager</a>, which means that applications do not draw directly to the framebuffer, but to a texture in video memory; the window manager then uses the GPU to display all windows on their respective positions. This is both more efficient (for example, dragging a window doesn't require all affected applications to redraw their content up to 60 times per second anymore) and visually pleasing (it prevents those <a href="https://msdn.microsoft.com/en-us/library/windows/desktop/ff684179%28v=vs.85%29.aspx">ugly broken windows</a> that appear when dragging a window over another window whose application is not responding to redraw calls anymore).</p>
<p>Of course, while compositing all those windows/textures on the GPU is very efficient, it is still not free; especially when only a single full-screen application like a video player is being displayed, compositing is only a waste of resources. "Unredirect output" seems to allow such full-screen applications to again draw directly to the framebuffer (as opposed to a texture). However, some applications seem to have problems doing just that; they somehow get their timing wrong and draw their window contents at the wrong moments (i.e. not during the "vertical sync period" a.k.a. VSync), which leads to visual tearing.</p>
<p>It turns out that Compiz (at least on my distribution, Ubuntu 14.10) already comes with a pre-populated list of applications that are excluded from unredirecting. That list explains why I was experiencing tearing in Chrome, but not in any other application: All video players and other applications I have tried are preloaded on that list – except Chrome or Chromium!</p>
<div class="highlight"><pre>(any) & !(class=Totem) & !(class=MPlayer) & !(class=Vlc) & !(class=Plugin-container) & !(class=Firefox)
</pre></div>
<p>Adding filter clauses for both Chrome and Chromium completely fixes the issue for me (the list can be accessed and modified in the "Composite" tab of the "CompizConfig Settings Manager", which should be available in your distribution if it ships with Compiz):</p>
<div class="highlight"><pre>[...] & !(class=^Google-chrome) & !(class=Chromium)
</pre></div>
<p>This fix should solve the problem regardless of your graphics card manufacturer; if you are using an Intel GPU, you might also have luck with enabling the <a href="https://wiki.archlinux.org/index.php/Intel_graphics#Tear-free_video">TearFree</a> option of the video driver, which might or might not be more efficient and/or cause other problems with your graphics – I have decided to use the Compiz fix, since it aligns with the way all other applications already are drawing their fullscreen windows on my system.</p>
<p>If you are experiencing the same problem, let me know if this fix helps in the comments!</p>vim and that weird one-second startup delay2014-05-15T20:16:00+02:00lxgrtag:blog.lxgr.net,2014-05-15:posts/2014/05/15/vim-tmux-startup-delay/<p>Are you using <code>vim</code>, <code>tmux</code>, a graphical Linux desktop and are you experiencing random sluggishness when starting your editor? If not, you can skip this one.</p>
<p>This is something that had been bugging me for ages, first at work on my workstation, then at home: Long-running <code>tmux</code> sessions would sporadically induce startup delays of the <code>vim</code> editor of exactly one second. Reattaching <code>tmux</code> didn't solve the problem; logging out and back into my desktop always did.</p>
<p>First I thought I was just being impatient, but after some profiling with <code>time</code>, I was getting curious. <code>strace</code> revealed that the delay was <em>exactly</em> one second: Something in <code>vim</code>s startup process was calling the <code>nanosleep(2)</code> system call with one second as an argument!</p>
<p>To make a long story short: This is caused by some X library that is mislead by a broken environment variable <code>SESSION_MANAGER</code> from a former X session. <code>tmux</code> tends to get rather attached to environment variables, which, in this instance, causes the sluggishness.</p>
<p>If the problem goes away after executing</p>
<div class="highlight"><pre>unset SESSION_MANAGER
</pre></div>
<p>or something similar for your shell, you can fix it permanently by appending the following line to your <code>.tmux.conf</code>:</p>
<div class="highlight"><pre>set-option -g -a update-environment " SESSION_MANAGER"
</pre></div>
<p>If you are now wondering why <code>vim</code> would need access to some X-related variables in the first place (as I was): It lets <code>vim</code> access your X clipboard! (Strangely, the variable <code>SESSION_MANAGER</code> is not actually needed for that, but you can verify it by overwriting some more critical X variable like <code>DISPLAY</code> or <code>XAUTHORITY</code> and subsequently trying to use the X clipboard from within <code>vim</code>.)</p>On agents and keychains (Part 3)2014-05-12T21:40:00+02:00lxgrtag:blog.lxgr.net,2014-05-12:posts/2014/05/12/on-agents-and-keychains-part3/<p>In the previous posts of this series, I've <a href="//blog.lxgr.net/posts/2014/05/10/on-agents-and-keychains-part1/">described the operating environment</a> of a password or private key agent and <a href="//blog.lxgr.net/posts/2014/05/11/on-agents-and-keychains-part2/">given a summary of their tasks</a>. This time, we'll see how some real-world agents are implemented.</p>
<p>But before that, a disclaimer: I'm merely an interested observer of all of the tools mentioned below. All my knowledge is from looking at their documentation, source code and from practical experiments. If you plan to use any of them for your private, sensitive data, you should definitely not rely solely on this analysis.</p>
<h1>Part 3: Real-world password and private key agents</h1>
<h2><code>ssh-agent</code></h2>
<p>The first tool we'll be looking at is my personal favorite of the batch: <a href="http://manpages.ubuntu.com/manpages/trusty/en/man1/ssh-agent.1.html"><code>ssh-agent</code></a>. Its job is to protect an user's private SSH authentication keys.</p>
<p>Usually, those keys are stored in the user's home directory, encrypted with a symmetric key derived from a passphrase that has to be entered every time the key is used to connect to a remote server using SSH; <code>ssh-agent</code> was developed to avoid having to type it that often.</p>
<p>When an instance of <code>ssh-agent</code> is started, it creates a <a href="https://en.wikipedia.org/wiki/Unix_domain_socket">Unix domain socket</a>; the file system path of that socket will usually be stored in an environment variable called <code>SSH_AUTH_SOCKET</code>. Starting the agent and setting the variable is usually handled by a few lines in the user's desktop and/or shell configuration files. This socket is then used by <code>ssh-agent</code>'s clients to request various operations.</p>
<p>First of all, to be of any use, the private keys have to be actually loaded into the memory of the agent. This is performed by a tool called <a href="http://manpages.ubuntu.com/manpages/trusty/en/man1/ssh-add.1.html"><code>ssh-add</code></a>, which basically asks the user for his passphrase and the location of his private keys, decrypts them in memory, and sends them over the Unix socket to the agent.</p>
<p>The nice thing about <code>ssh-agent</code>'s <a href="http://www.openbsd.org/cgi-bin/cvsweb/src/usr.bin/ssh/PROTOCOL.agent?rev=HEAD;content-type=text%2Fplain">protocol</a> spoken over the socket is this: There is no command to extract a private key from it. Clients (mostly instances of the SSH client, <code>ssh</code>, really) can request the agent to sign some data on their behalf, which in turn allows them to authenticate against a remote SSH server. There are some other commands (e.g. to remove some or all private keys, temporarily lock or unlock the agent with a password, or get a list of currently loaded keys), but except for security bugs or other side channels, there is no way to make the agent reveal the private keys.</p>
<p>As I've mentioned, a security tool can only be as secure as the environment it's running in and on whose security measures it is relying on. In the case of <code>ssh-agent</code>, this is the user's Unix account, and in many cases some graphical desktop environment. <code>ssh-agent</code> (or at least the version of OpenSSH included in Ubuntu) tries to limit the ways other applications in the same context can interact with its virtual memory by <a href="http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/trusty/openssh/trusty/view/head:/ssh-agent.c#L1154">disabling</a> the <code>ptrace(2)</code> facility of the operating system. I'll write a lot more on that in a future post, but for now, it suffices to say that this (hopefully) makes it impossible for other processes to peek into an agent's memory space (using <code>gdb</code> or the <code>/proc/<pid>/mem</code> device).</p>
<h2><code>gpg-agent</code></h2>
<p>The next tool on our list seems to be quite similar to <code>ssh-agent</code>: <a href="https://www.gnupg.org/documentation/manuals/gnupg/Invoking-GPG_002dAGENT.html"><code>gpg-agent</code></a> also protects private keys, uses a Unix socket and an environment variable to answer to requests, and runs with the user's permissions, started in one of the various startup scripts of the desktop environment. It is used to protect a user's private (or secret) <a href="https://www.gnupg.org/">GnuPG</a> encryption and signature keys.</p>
<p>Unfortunately, the similarity ends when it comes to how <code>gpg-agent</code> protects the user's private keys. In fact, I couldn't believe my eyes when I ran <code>strace</code> on an instance of <code>gpg</code> while executing a private-key operation:</p>
<div class="highlight"><pre><span class="gp">$</span> strace gpg --armor --gen-revoke 2F5BBF5C
<span class="go">write(8, "GET_PASSPHRASE 1AA19BADB016B8BF3"..., 203) = 203</span>
<span class="gp">#</span> <span class="o">[</span>...<span class="o">]</span>
<span class="go">read(8, "OK 70617373776F7264", 1002) = 19</span>
</pre></div>
<p><code>gpg-agent</code> is not a private key agent at all! It merely caches the private key passphrase, handing it out to anyone asking niecly over the Unix domain socket (which means every application running with the user's privileges). This negates almost all of the security benefits of using an agent in the first place, and on top of that, it is even <em>less</em> secure than just storing the private key in the agent's memory: If the user's session is compromised, not only the private key, but also the passphrase can be recovered by an attacker.</p>
<p>I can only guess that there are various historical reasons for <code>gpg-agent</code>'s architecture, but <code>ssh-agent</code> shows that there is a better way to handle private key caching in the userspace.</p>
<h2>GNOME Keyring</h2>
<p>GNOME is used as the default desktop environment for many Linux distributions; and even more are using only some parts while providing their own user interface (window manager, compositor, default applicatoins etc.) – Ubuntu is a famous example of the latter category.</p>
<p>The GNOME applications include a handy tool called <a href="https://wiki.gnome.org/Projects/GnomeKeyring/">GNOME Keyring</a>, which is primarily a password manager, but can also act as a private-key manager for both SSH and GnuPG. I'm not using its private-key features any more for various reasons, but it is still my password manager of choice for everything else (primarily for web browsers).</p>
<p>The documentation page of the software is <a href="https://wiki.gnome.org/Projects/GnomeKeyring/SecurityPhilosophy">very upfront</a> about what the tool can and cannot achieve: The developers openly state that for the current desktop architecture, secure privilege separation between applications is simply not possible.</p>
<p>There used to be some kind of access control system for applications, but as it is now, every application running with the user's privileges can store and request plaintext passwords to and from the agent, which in turn stores them in an encrypted database in the user's home directory.</p>
<p>The encryption key is derived from a passphrase defined by the user; if it is equal to their Unix login password, the keychain is conveniently "unlocked" (i.e., the encryption key is stored in the Keyring daemon's memory) as soon as the user logs in to their desktop.</p>
<p><code>gnome-keyring</code> doesn't perform any security theater to make it seem as if the passwords of an unlocked keyring were somehow more secure than they are, but like <code>ssh-agent</code>, its memory is protected from access by other user processes on my system (because its binary has a <a href="http://manpages.ubuntu.com/manpages/trusty/man7/capabilities.7.html">Linux capability</a> enabled).</p>
<h2>OS X Keychain</h2>
<p>Like GNOME, OS X provides users with a way to securely store their passwords on the disk while still granting automatic access to other applications as long as the user is logged in: <a href="https://developer.apple.com/library/mac/documentation/security/conceptual/keychainServConcepts/02concepts/concepts.html">OS X Keychain</a>. However, it aims to go even further than that: By using an ACL based on the code signature or binary hash of requesting programs, it <a href="https://developer.apple.com/library/mac/documentation/security/conceptual/keychainServConcepts/02concepts/concepts.html#//apple_ref/doc/uid/TP30000897-CH204-CJBIBIBC">restricts access</a> to the stored password to a subset of all applications running with the user's permissions.</p>
<p>The OS X desktop environment, at least theoretically, also seems to implement more security measures than X11: Applications do not seem to be able to <a href="https://developer.apple.com/library/mac/documentation/cocoa/conceptual/eventoverview/MonitoringEvents/MonitoringEvents.html">install global keyloggers</a> without first requesting special permissions from the user and the <code>ptrace</code> system call (or its cousin, <code>task_for_pid</code>, as it is known on OS X) is only available to <a href="https://blogs.oracle.com/dns/entry/understanding_the_authorization_framework_on">privileged users</a> or <a href="http://wiki.lazarus.freepascal.org/GDB_on_OS_X_Mavericks_and_Xcode_5">signed debugging tools</a> (which in turn require user authentication).</p>
<p>Additionally, the Keychain service seems to be running with superuser privileges, so it might theoretically be able to perform some additional verifications of the process requesting a password (maybe the aforementioned checks of the binary hash and/or code signature).</p>
<p>But is that really enough to protect all potentially malicious, <a href="http://juusosalonen.com/post/30923743427/breaking-into-the-os-x-keychain">non-root</a> accesses to the stored passwords? That will be the topic of the article concluding this little series, but before that, we'll see how process memory isolation of binaries running with the same user permissions could possibly be achieved.</p>On agents and keychains (Part 2)2014-05-11T13:19:00+02:00lxgrtag:blog.lxgr.net,2014-05-11:posts/2014/05/11/on-agents-and-keychains-part2/<p>In the <a href="//blog.lxgr.net/posts/2014/05/10/on-agents-and-keychains-part1/">previous post</a> of this series, I've roughly described the operating environment of a password or private key agent; this time, I'll try to summarize the basic structure and tasks of such an agent.</p>
<h1>Part 2: What does an agent do?</h1>
<p>The job of a password or private key agent is to protect, but also to share, secrets. In the case of a password manager, the secrets are the plaintext authentication tokens, or passwords, for various user accounts and services – e.g. web and mail passwords, Wi-Fi preshared keys and many more; a private key agent like <code>ssh-agent</code> or <code>gpg-agent</code> protects one or more private keys used for signing and/or encryption of messages or for the use in authentication protocols.</p>
<p>The entities that the secrets are (or are not) shared with are other processes running on the user's computer, running with the permissions of his user identity.</p>
<h2>Restricting access to internal state</h2>
<p>In order to be able to actually protect a user's secrets, the agent has to have some way to actually keep secrets from other processes. At least on a classical Unix system, this is a difficult task when running with the same permissions as those processes: As I've tried to explain in the last article, the permission model is not really designed for privilege separation of user applications.</p>
<p>In fact, the task at hand might be better solved by a classical Unix daemon; a trusted server process running under a different user ID (either the superuser's or one specifically created for the daemon). (In fact, this is the approach Apple took for their OS X Keyring.)</p>
<p>However, this is not the approach taken by some popular agents – but that's actually the topic of another article in this series.</p>
<h2>Sharing secrets with trusted applications</h2>
<p>While keeping secrets is an important task for an agent, there has to be some way for trusted applications to gain direct (in the case of passwords) or indirect (for private keys) access to those secrets – an agent that simply keeps all the secrets to itself is perfectly secure, but also perfectly useless.</p>
<p>The concept of a "trusted application" is trickier than it might seem: How would the trustworthiness of an application be defined in the first place? One might be tempted to enumerate a set of such trusted applications, e.g. the web browser(s) of a user for web authentication passwords, their mail client for their mail passwords and so on. But how does the agent actually identify the requesting entity?</p>
<p>Any approaches based on heuristics like the name of the executable file of the requesting process don't work: Users can install binaries in their home directory, where they can be freely replaced or modified by attackers.</p>
<p>A more sophisticated way to ensure integrity of a requesting process would be to hash the contents of its executable file and store that hash in the agent along with the secrets. This doesn't seem trivial to do, at least in the user space – on Linux, the agent would probably have to work with the <code>/proc</code> virtual file system to identify the executable of a process, but any such checks would be very likely be susceptible to <a href="https://en.wikipedia.org/wiki/Time_of_check_to_time_of_use">TOCTOU</a> vulnerabilities. The operating system might theoretically be able to provide the agent with a trustworthy hash of a requestor, though – but I suspect that this is not possible on the stock Linux kernel, for example.</p>
<p>An alternative to hashing the requesting binary is code signing. Operating systems that allow executable files to be signed by their developers or some other entity could provide the agent with the identity of the signer, which would allow "safe" modifications of the requester by its original developer or a system administrator (e.g. software updates or security patches).</p>
<p>Unfortunately, even if the authenticity of the requestor could be determined beyond doubt, this is still not enough: What if the attacker is able to coerce an otherwise trusted application to make a request on their behalf, and reveal the reply to them? This scenario will be part of another future article.</p>
<h2>Limitations to the agent model</h2>
<p>Before examining some agents in detail, it should be said that there are some fundamental limitations to what even a perfectly secure agent can achieve. If the user's machine (or even only his own account) is compromised in a global way, e.g. by an attacker that installs a key logger or is able to remotely control a user's session, the security benefits of the agent might very well be completely negated.</p>
<p>In other words, it is probably sufficient for an agent to be as secure as the environment it is operating in. An OS that does not provide even basic application isolation for graphical applications is very hard to protect indeed. (Unfortunately, almost all X11-based desktop environments are all but impossible to secure against untrustworthy applications.)</p>
<p>Even then, an agent might be of limited utility on such an untrustworthy system, if its task is to only indirectly grant access to sensitive data. This is exactly the situation for an SSH or GPG agent: By design, such an agent will never expose a user's private keys, but will only execute various private key operations on requestor-provided data items.</p>
<p>While an attacker with user privileges will be able to execute a number of such operations (e.g. logins on remote servers, decryption of single messages), after the compromise is detected, the keys do not necessarily have to be changed or revoked. (This might not be the case for some applications, though – e.g., if it is possible to sign new public keys with an existing private key and mark them a originating from the same user.)</p>On agents and keychains (Part 1)2014-05-10T14:22:00+02:00lxgrtag:blog.lxgr.net,2014-05-10:posts/2014/05/10/on-agents-and-keychains-part1/<p>Many people, myself included, use tools like ssh-agent or gpg-agent to protect their private keys from theft without sacrificing the convenience of password-less logins. Presumably even more people use some kind of password manager, whether that is the one included with their operating system or a third-party one. I've been using both for a long time, but only recently started to wonder about their internals: What is the threat model here, and how do those tools provide the necessary protection? This will be a series of posts on the subject; in this one, I will try to examine the necessity of such programs, and the way process separation is implemented in various operating systems.</p>
<h1>Part 1: The need for application isolation</h1>
<h2>Isolate what?</h2>
<p>Before looking at the tools in question, we have to look at the environment in which they are being used.
The various Unix operating systems have historically focused their security efforts on separating the actions of multiple users on a single system – i.e., Alice is not supposed to be able to read Bob's mail. Processes usually run with a user's permissions and are free to read and modify files in their home directory, as well as communicate with each other almost without restrictions (more on that later).</p>
<p>This model does not distinguish between a user and the programs he is using. Whether a user runs a simple unix command like <code>mv</code> or a complex application like a web browser, the operating sytem kernel assumes that all the system calls by the user's processes are identical with the user's intentions.</p>
<p>While this assumption is still reasonable if all of the binaries our user might run are provided by the same people who provide their operating system, things start to get interesting once users bring their own software, whether voluntarily or accidentially (in the form of malware received through whatever vector).</p>
<p>Once the user runs any piece of "evil" code, they lose. The operating system will still isolate their requests from other users on the system, but that might be little consolation for the case of a typical desktop user – more often than not, they are the only user on their system (at least as far as Unix permissions are concerned), and a malicios application running with their permissions amounts to a full system compromise.</p>
<p>At a first glance, it seems to be impossible for an user of such a system to protect some piece of information against their own processes, but still, the existance of password managers that use anything but a plain-text unencrypted database seems to indicate that at least their vendors think (or try to convince their customers to think) otherwise.</p>
<h2>A different model</h2>
<p>Before looking at the tools in question, I think it is interesting to examine some other ways of application privilege separation.</p>
<p>While most desktop systems essentially still operate under the same security paradigms, the situation is very different for the mobile operating systems. (Presumably) inspired by the situation of malware on the most common desktop operating systems, their creators have realized that in order to unleash an enormous whealth of third-party applications on their users by design, a more strict separation of privileges is in order.</p>
<p>Android has implemented application separation in a simple, yet very effective way: Every Android app has a unique Unix user account. This way, by default all application data is implicitly private. In order to use anything but their own data, applications have to use Android's library functions that will moderate access to system functions and potentially sensitive data. (Permissions to use those library functions are granted at the time an application is installed in an all-or-nothing fashion, but that is a design decision that could be modified to a more fine-grained model pretty easily.)</p>
<p>Apple's iOS uses a more traditional approach with regards to UIDs - all applications are running as the user <code>mobile</code>. Sandboxing is instead explicitly implemented in the kernel, which restricts each application's system calls to a secure subset. Basically, reads and writes to anything but a list of allowed files and directories and other security-critical system calls will fail. Some exceptions to this can be granted by the user at run-time (e.g. access to the address book or location based services).</p>
<h2>Apps on the desktop?</h2>
<p>Can such a security model be brought to the desktop without breaking almost any existing application? It seems that at least Apple thinks that this is the way to go. While it is still possible to run unrestricted applications on OS X, there is now also a sandboxing mechanism in place that allows a developer to whitelist the set of allowed system calls just like on iOS. This will presumably become a mandatory feature of new applications submitted to the Apple-curated Mac app store. Microsoft seems to be trying to do something similar with their Windows Store.</p>
<p>While this might solve the problem of application isolation, there are a lot of legacy applications that will probably be never ported to such a restrictive environment, and the third popular desktop operating system, Linux, does not provide such a sandboxed app-store model for obvious reasons.</p>
<p>Still, there are password managers and private key agents for all three major platforms – (how) do they work? This will be the topic for the following articles.</p>ssh-agent and the OS X Keychain2014-05-08T16:06:00+02:00lxgrtag:blog.lxgr.net,2014-05-08:posts/2014/05/08/ssh-agent-osx-keychain/<p>Are you relying on OS X's Keychain to protect your SSH key passphrases? You
shouldn't. (The "plain" ssh-agent is fine, though.)</p>
<p>To be continued!</p>How to fix slow DNS lookups on Ubuntu2013-11-18T22:03:00+01:00lxgrtag:blog.lxgr.net,2013-11-18:posts/2013/11/18/nsswitch-ubuntu-slow-dns-lookups/<p>If you're using a relatively recent version of Ubuntu, chances are that you have encountered spurious slowdowns that might be related to a very specific DNS failure. For me, it was the fact that <code>ping</code> to a host <em>without</em> a reverse DNS entry would only transmit a single ICMP request per second, even when a higher rate was specified via the <code>-i</code> option.</p>
<p>I've traced the DNS requests that are performed by <code>ping</code> by default (the effect does not occur when using the <code>-n</code> option which disables host name lookup), and didn't notice anything out of the ordinary. The <code>NXDOMAIN</code> responses were occuring almost instantly, but nevertheless, it took precisely one second for this response to actually propagate to the <code>ping</code> process.</p>
<p>To make a long story short: The reson for this is that Ubuntu (or more precisely, the Name Switching Service), will (by default) try to lookup DNS records not only via the regular DNS server configured via the network settings, but also using Zeroconf (a.k.a. Bonjour); a protocol that can be used to resolve hostnames locally by using multicast DNS requests and responses.</p>
<p>This is not an issue for DNS queries that can be answered positively by your regular DNS server (those will always take precedence over records received via Zeroconf), but it can be a problem for negative DNS responses (<code>NXDOMAIN</code>): When the resolving library receives one of those, it will try a Zeroconf lookup, and this can take a while – especially for a host that does not exist.</p>
<p>Since Zeroconf is only rarely used on Linux and is almost always limited to the <code>.local</code> top-level domain, this behavior seems useless at best, and can be pretty irritating.</p>
<p>To fix it, you can simply disable the Zeroconf DNS lookups in the configuration file <a href="http://man7.org/linux/man-pages/man5/nsswitch.conf.5.html"><code>/etc/nsswitch.conf</code></a> by changing the line</p>
<div class="highlight"><pre><span class="n">hosts</span><span class="o">:</span> <span class="n">files</span> <span class="n">myhostname</span> <span class="n">mdns4_minimal</span> <span class="o">[</span><span class="n">NOTFOUND</span><span class="o">=</span><span class="k">return</span><span class="o">]</span> <span class="n">dns</span> <span class="n">mdns4</span>
</pre></div>
<p>to</p>
<div class="highlight"><pre><span class="n">hosts</span><span class="o">:</span> <span class="n">files</span> <span class="n">myhostname</span> <span class="n">mdns4_minimal</span> <span class="o">[</span><span class="n">NOTFOUND</span><span class="o">=</span><span class="k">return</span><span class="o">]</span> <span class="n">dns</span>
</pre></div>
<p>This doesn't entirely disable Zeroconf – it only restricts the lookups to the <code>.local</code> domain, which is almost always the only place where they are useful anyway.</p>
<p>The effects should be immediately noticeable – just try to ping one of the previously slow to respond hosts and check if the ICMP requests are still limited to one per second.</p>
<p>If you think that this should be the default configuration for Ubuntu, you are not alone – there <a href="https://bugs.launchpad.net/ubuntu/+source/nss-mdns/+bug/94940">is a bug report</a> on Ubuntu's bug tracker that describes the problem, but since it's been known since 2007, I wouldn't bet on the default changing anytime soon.</p>Thoughts on a cloud-based password synchronization service2013-10-24T16:15:00+02:00lxgrtag:blog.lxgr.net,2013-10-24:posts/2013/10/24/thoughts-on-cloud-password-sync/<p>Today, Apple has enabled its cloud-based password synchronization service, iCloud Keychain. The service promises to safely store and synchronize passwords and other sensitive user data like credit card numbers among multiple devices. Apple claims that the information is protected with AES, but that alone is meaningless without knowing where that key is actually stored.</p>
<p>As usual, there is not much public documentation, but there is a <a href="http://support.apple.com/kb/HT5813?viewlocale=en_US">support document</a> that contains some interesting propositions:</p>
<ul>
<li>Adding a new device to an iCloud-synchronized Keychain displays a message on a previously registered device to accept or deny that new device. </li>
<li>When enabling Keychain sync, the user is given an option to create a backup code.</li>
<li>With the backup code, it's possible to recover the Keychain contents without the original device; without the code, (supposedly) not even Apple can access the contents.</li>
<li>The number of times a user can enter the security code is limited; Apple support can extend the limit, but eventually, the Keychain data will be deleted from the server.</li>
</ul>
<p>Starting from those propositions alone, I was wondering how it might be possible to implement a password storage and synchronization service that has all those properties. Is there a way to enable such a service without simply storing the AES key on the servers, and using the user password to retrieve it together with the data? The following is based on speculation alone; I haven't done any reverse engineering on the actual Keychain software or protocol.</p>
<p>The first statement about adding a new device sounds like there is some kind of key exchange going on, which the user can allow or deny. The new device could present a public key to the original device, and the old device could then encrypt the AES key with that public key. (Every iOS device already has at least one RSA key in the form of a certificate signed by Apple's certificate authority.) Without any kind of fingerprint verification, there is no way to verify that the public key actually belongs to the new device and not to some third party, though. </p>
<p>Disregarding any possible MITM attacks on the key exchange, this way of adding new devices could be used to safely share the password database and its encryption key among many devices. The shared key can also be used for efficient synchronization of future modifications to the database.</p>
<p>The second and third statements about the backup code sounds like a way to store a copy of the database encryption key on the server, which might be wrapped with yet another key derived from the backup code. The default strength of the backup code is only a four-digit number, which even when used with PBKDF2 with many iterations is barely more secure than plaintext, but it can be changed to a more secure alphanumeric password. When using a reasonably secure passphrase, this makes it impossible for the service provider to access the database contents.</p>
<p>The fourth claim about a limit to the number of attempts to enter the backup code could be implemented with a secure hash function. When the backup code is first created, it is not only used as an input to a key derivation function which is then used to wrap the database encryption key before it is sent to the server, but also hashed (optionally with a salt and a number of iterations). The resulting hash is also transmitted to the server together with the wrapped backup key.</p>
<p>When the user later initiates a database restore, the server first transmits the salt (if there is one) to the client. The user then enters the backup code on the device, where it is hashed with the salt, and transmitted back to the server. Only when the response to that challenge is identical to the response stored on the server, the actual database will be sent to the client in its encrypted form. This way, the number of backup code attempts per second can be rate-limited on the server side.</p>
<p>This would make it possible to prevent brute-force attacks on a weak backup code for other clients. Of course, it doesn't help against an untrustworthy service provider, who will be able to brute-force the encryption key without any limitation, since he necessarily holds the backup copy of the database and its wrapped encryption key.</p>
<p>I would be very interested in a detailed protocol analysis of Apple's solution, like the one that was recently published about the iMessage protocol. Using an architecture like the one lined out above would put Apple in a similar position as for iMessage with regards to lawful interception: While government access would be possible via a MITM-attack on the device setup procedure, it wouldn't be as simple as demanding the user database and the according encryption key. Everything else would more or less invalidate the unambiguous statement (as quoted from the <a href="http://support.apple.com/kb/HT5813?viewlocale=en_US">support page for iCloud Keychain</a>) regarding Apple's capabilities: "If you choose to not create an iCloud Security Code, Apple will not be able to recover your iCloud Keychain."</p>
<p>Of course, if a user choses to use a four-digit numeric backup code (which is the proposed default by the setup wizard), the details of the implementation are rendered moot: There is no way such a weak password can provide any security against brute force attacks using any practical combination of hash function and iteration count. (This is probably also the reason why the service implements a rate-limiting feature for recovery access.) It would have been in the interest of Apple's user base to provide a strong, randomly generated alphanumeric string as a backup code by default, <a href="https://support.mozilla.org/en-US/kb/firefox-sync-data-secure-find-out-more">like Mozilla does</a> for their bookmark synchronization service.</p>
<p><strong>Update (2013-10-30):</strong>
Ars Technica has published an <a href="http://arstechnica.com/information-technology/2013/10/apple-claim-that-icloud-can-store-passwords-only-locally-seems-to-be-false/">interesting article</a> on the topic, with similar conclusions. They claim that there is a different recovery process depending on whether a four-digit security code or an actual high-entropy password is used, which is somewhat strange (if there is really no server-side brute-force protection for alphanumeric passwords, a four-digit passcode could actually provide better protection than a five-character alphanumeric password). Using a high-entropy password seems like the better choice in any case.</p>Safe deterministic (EC)DSA signatures are coming to OpenSSL2013-08-30T00:37:00+02:00lxgrtag:blog.lxgr.net,2013-08-30:posts/2013/08/30/deterministic-dsa-signatures-openssl/<p>By now, everybody involved in implementing algorithms using the DSA or the ECDSA signature schemes should <em>really</em> understand <a href="http://tools.ietf.org/html/rfc6979">the importance of a proper secret nonce</a> as one of the inputs for a signature.</p>
<p>This is easy to get wrong, both because PRNGs <a href="http://www.debian.org/security/2008/dsa-1571">are</a> <a href="http://kakaroto.homelinux.net/2012/01/how-the-ecdsa-algorithm-works/">really</a>, <a href="http://jbp.io/2013/08/15/android-securerandom-guess/">really</a>, <a href="http://android-developers.blogspot.com/2013/08/some-securerandom-thoughts.html">really</a> <a href="http://armoredbarista.blogspot.com/2013/03/randomly-failed-weaknesses-in-java.html">hard</a> to get right, and because not everybody implementing/using (EC)DSA expected to be needing randomness just for <em>signing stuff</em> (as opposed to creating key pairs).</p>
<p>Fortunately, there is a way out. <a href="http://crypto.stackexchange.com/users/452/poncho">Poncho on Stackexchange Crypto</a> has notified me about <a href="http://tools.ietf.org/html/rfc6979">an interesting RFC</a> in the comments on a <a href="http://crypto.stackexchange.com/a/9939/2538">nice answer</a> to a related question.</p>
<p>The really clever idea is that there is another way to (probabilistically) ensure that a secret nonce is used for every signature than just using a PRNG and hoping for the best.</p>
<p>Since reusing the same nonce for <em>the same message</em> signed by the same key will always give the same signature as an output (there are no other inputs to the signature algorithm), we just have to guarantee that the nonce is different and unpredictable for <em>different</em> messages.</p>
<p>By using a hash of the message and the private key as the nonce, these conditions can be satisfied even without a proper PRNG. Even better, it's possible to hash them together with some random data to provide backwards compatibility to implementations that react badly to deterministic (EC)DSA signatures. (Maybe some regression tests might interpret the lack of randomness as a fatal design flaw.)</p>
<p>An (informal) RFC is nice, but actual code is even nicer, so I'm very happy that <a href="http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=190c615d4398cc6c8b61eb7881d7409314529a75">a patch</a> implementing this method and <em>making it the default</em> in OpenSSL has already been accepted to the development version.</p>
<p>If you're interested in the details, there's <a href="https://www.imperialviolet.org/2013/06/15/suddendeathentropy.html">a blog post</a> by the author that has some more details.</p>
<p>I'm really looking forward to this patch shipping in a lot of OpenSSL binaries, whether as a part of a distribution or embedded in some other software – there have been more than enough fatal PRNG(EC)DSA failures in the past for my liking.</p>TLS client certificates and Mobile Safari2013-08-27T16:02:00+02:00lxgrtag:blog.lxgr.net,2013-08-27:posts/2013/08/27/tls-client-certs-safari/<p><strong>Update (2013-08-31):</strong> Apple has asked me to refrain from publishing any details on this security-relevant bug for the time being; I hope that a fix will be released soon. When that happens (or after a reasonable amount of time has passed), the original post will be restored.</p>
<p>Until then, I would strongly advise against using Mobile Safari when any X.509 client certificates are stored on an iOS device, e.g. an S/MIME encryption/signing certificate. Other in iOS, like Chrome, are not affected; neither are browsers on OS X (including Safari).</p>
<p><strong>Second update (2013-10-23):</strong> Since my original post, iOS 7 has been released; the bug described below seems to have been fixed. The issue is of course still present in iOS <= 6.1.4. Since it seems to be Apple's policy not to release security fixes for discontinued OS versions, this leaves older devices like the original iPad and the iPod touch (up to the 4th generation) vulnerable. That's unfortunate, but since I'm definitely not the only one who knows about the issue, here is my original post. Be sure to take care when using any client certificates on an older iOS device.</p>
<p><strong>tl;dr:</strong> If you have an S/MIME or other X.509 client certificate installed on your iOS device, Mobile Safari will hand it out to any web server that asks for it – without asking you.</p>
<p>Recently, I've looked into TLS with client certificates, specifically into how the various browsers and operating systems implement them.</p>
<p>In addition to authenticating a server and securing a connection between this server and an anonymous client, TLS also allows the client to identify itself to the server using its own X.509 certificate. This mode is only used by very few services using TLS, which could be attributed to the difficulty of issuing client certificates in the first place, and protecting them against both theft and loss later on.</p>
<p>However, I think that there are more issues with client certificates than that.</p>
<p>First of all, the client certificate is transmitted to the server <a href="https://tools.ietf.org/html/rfc5246#section-7.4.4">unencrypted</a>, which means that everybody between the client and the server is able to identify the user trying to connect. Since an X.509 certificate frequently contains personal information like the user's full name and mail address,this seems like a bad thing to do.</p>
<p>Additionally, TLS client certificates are used in a way that doesn't provide <a href="http://en.wikipedia.org/wiki/Deniable_authentication">deniable authentication</a>. To prove that the client is in posession of the private key corresponding to the X.509 certificate, he <a href="https://tools.ietf.org/html/rfc5246#section-7.4.8">signs all previous handshake messages</a>. Among other things, this contains a (client-provided) timestamp and the server certificate; the signature of those values <a href="http://crypto.stackexchange.com/questions/5455/does-a-trace-of-ssl-packets-provide-a-proof-of-data-authenticity">can be used to prove</a> that that somebody with access to the private key initiated a connection to a specific server at a specific time. Even worse, this signature is also still transmitted in plaintext (symmetric encryption and authentication aren't used before the next message (Finished) in the handshake.</p>
<p>Considering those (in my opinion substantial) disadvatages of the implementation of client certificate authentication in the current version of TLS, it might be better to perform authentication inside the secure TLS channel at the application layer, which is exactly how it's done for the vast majority of web services (via HTTP cookies) and other protocols protected by TLS.</p>
<p>(An even better solution would be a TLS extension that moves the client authentication inside the secure channel, or even uses something analogous to the server authentication in TLS, which might be able to provide deniable authentication for the client as well. But the rate at which TLS extensions and updates are adopted by software vendors is not exactly instantaneous.)</p>
<p>Since the status quo seems to be exactly that (whether that's due to the difficulty of issuing certificates or to the mentioned disadvantages of them with TLS), is there anything left to worry about?</p>
<p>There is: Broken browsers.</p>
<p>Probably due to their minimal use in real-word applications, some browsers' TLS client certificate implementations are a bit sloppy. When an HTTP server requests a client certificate (using the <a href="https://tools.ietf.org/html/rfc5246#section-7.4.4">Certificate Request</a> message in the TLS handshake), most of them display a pretty technical-looking dialog to the user, who might or might not understand what's going on.</p>
<p><img alt="Chrome's client certificate selection dialog" src="//blog.lxgr.net/images/chromecert.png" /></p>
<p>This is clearly not an example of good user experience. So let's check how Apple does it in iOS...</p>
<p><img alt="Mobile Safari's lack of a certificate selection dialog" src="//blog.lxgr.net/images/ioscert.png" /></p>
<p>Oops. They don't. They just pick the first certificate available (in my case, this is an S/MIME certificate that includes my full name, my employer and my e-mail address), transmit it and authenticate to the server by non-repudiably signing the TLS handshake – all in plaintext. All the previously mentioned caveats apply, only that the user has no choice about the matter in the first place.</p>
<p>If you want to try it yourself, just visit <a href="https://www.mikestoolbox.net/">Mike's Toolbox</a> with Mobile Safari, accept the self-signed server certificate and look for your name or e-mail address on that page.</p>
<p>This problem has been mentioned before publicly <a href="http://forums.whirlpool.net.au/archive/1936101">at least once</a>, more than one year and one major OS version ago. On the desktop, this has already been fixed (with <a href="http://support.apple.com/kb/HT1679">OS X 10.5.3</a>); I'm really hoping it will be fixed with iOS 7 as well.</p>Android's SecureRandom - not even nonce2013-08-15T15:30:00+02:00lxgrtag:blog.lxgr.net,2013-08-15:posts/2013/08/15/android-securerandom-not-even-nonce/<p>There has been a bit of drama about the <a href="https://bitcointalk.org/index.php?topic=271486.0">theft of some 55 Bitcoins</a> (worth about $5500 at the current exchange rate), with the common denominator that all of the corresponding private keys were stored in Android wallets. While this is not nearly the first case of Bitcoin theft, it is probably the first one that is a direct result of a crypto bug.</p>
<p>In this case, the problem resulted from the (re)use of the nonce used in the elliptic curve signatures that are used to generate Bitcoin transactions. As everybody familiar with (or even implementing) ECC-based encryption schemes should know well by know, reusing the signature nonce, or using a predictable value even once, <a href="http://blog.cryptographyengineering.com/2012/03/surviving-bad-rng.html">results in catastrophic failure</a>: The private key can then be trivially calculated from the signature(s). (If you don't believe me, just ask Sony...) This seems to be the method that was used to steal the Bitcoins in question.</p>
<p>So far, so bad. The obvious question now is: Who was responsible for reusing the nonces in the first place? Since the flaw is not limited to a single wallet implementation, but only occurs on Android (even though some of the Bitcoin libraries are also used on desktop bitcoin clients), people quickly came to the conclusion that there must be a flaw in one of Android's cryptographic libraries.</p>
<p>In <a href="http://permalink.gmane.org/gmane.comp.bitcoin.devel/2714">an announcement</a> to the Bitcoin dev mailing list, Mike Hearn, the developer of the Java library bitcoinj announced that the offender in question is the class <code>SecureRandom</code> of the Android framework. The various wallets for Android were quickly patched to avoid that class and use <code>/dev/urandom</code> directly, and as far as their developers and users are concerned, the problem is now solved.</p>
<p>However, when there is a bug in a security primitive implemented in a such widely used library, chances are that other users are also affected. So what exactly went wrong, and what are the implications?</p>
<p>Shortly after the announcement of the bug, people were quick to point to <a href="http://armoredbarista.blogspot.com.au/2013/03/randomly-failed-weaknesses-in-java.html">a paper discussing several vulnerabilities</a> of the <code>SecureRandom</code> implementations of various Java frameworks, among them Apache Harmony, which is the base for Google's Android framework. </p>
<p>Indeed, Android <a href="http://androidxref.com/4.3_r2.1/xref/libcore/luni/src/main/java/org/apache/harmony/security/provider/crypto/SHA1PRNG_SecureRandomImpl.java">uses that implementation</a> - but only up to and including version 4.1. Additionally, according to the paper, the flaw limits the entropy of an instance of <code>SecureRandom</code> to 64 bits of entropy. This is not enough for cryptographic applications like key generation or nonces, but also doesn't explain why in many cases, the exact same values were generated. <a href="http://en.wikipedia.org/wiki/Birthday_problem">On average</a>, 2^32 transactions would have to be generated to yield a single collision – all of those with the same key, e.g. bitcoin address.</p>
<p>Another problem of the Harmony implementation of <code>SecureRandom</code> is that using <code>setSeed()</code> on an instace replaces the existing entropy in the generator (instead of being mixed securely combined with it). When used wrongly (e.g. by seeding with not-so-random data), this could also lead to predictable values generated by the instance. (This behavior was even used by some applications to use <code>SecureRandom</code> together with a deterministic seed as some kind of key storage facility. Madness, I know...)</p>
<p>With Android 4.2, Google <a href="http://android-developers.blogspot.co.at/2013/02/security-enhancements-in-jelly-bean.html">finally switched</a> to <a href="http://androidxref.com/4.3_r2.1/xref/libcore/luni/src/main/java/org/apache/harmony/xnet/provider/jsse/OpenSSLRandom.java">a different implementation based on OpenSSL</a>. Since then, calls to the <code>setSeed()</code> method augment the internal entropy, instead of replacing it, as it should be. The new generator is even used when specifically asking for the Harmony-based one; obviously somebody at Google regarded the flaws important enough to allow modifying the behavior of legacy apps. (Of course, <a href="http://android-developers.blogspot.co.at/2013/02/using-cryptography-to-store-credentials.html">this broke all off-label usages</a> of <code>SecureRandom</code> in the process).</p>
<p>So, Android versions from 4.2 should be safe, right? As it turns out, <a href="http://android-developers.blogspot.co.at/2013/08/some-SecureRandom-thoughts.html">they are not</a>. In some circumstances (I haven't looked at the source yet), the new <code>SecureRandom</code> implementation goes horribly, horribly wrong and returns identical values for successive invocations. This is obviously very bad, not only for Bitcoin wallets, but basically for everybody using the Java cryptographic operations. According to the Android devs, that includes Key generation of symmetric and asymmetric keys (using the <code>KeyGenerator</code>, <code>KeyPairGenerator</code> and <code>KeyAgreement</code> classes).</p>
<p>Ironically, using <code>setSeed()</code> with proper random data with the OpenSSL implementation avoids the bug, which leads to the interesting situation that using <code>setSeed()</code> is discouraged for Android 4.1 and earlier, but is essential for 4.2. (Android 4.3 seems to avoid both bugs, according to the code in the blog post). The Android developers were kind enough to provide a ready-to-use mitigation in the form of a drop-in replacement for <code>SecureRandom</code> that does exactly that. (You should probably still be careful for older Android versions using the Harmony implementation – all the caveats for that mentioned in the paper still apply.)</p>
<p>Every Android developer using one of the affected classes should evaluate their usage, implement <a href="http://android-developers.blogspot.co.at/2013/08/some-SecureRandom-thoughts.html">the workaround</a> and necessary countermeasures, which could include warning your users to replace any keys generated with the weak random number generator, which is exactly what the developer of Bitcoin Wallet for Android <a href="https://code.google.com/p/bitcoin-wallet/source/detail?r=d9f996e218dd50c58855872c761e4acdd1ea1d15">did</a> promptly after learning about the vulnerability (the update generates a new, secure Bitcoin address and immediately transfers all funds to it after the update). If you don't trust any of the Android implementations of <code>SecureRandom</code> anymore, you should take a look at <a href="https://code.google.com/p/bitcoin-wallet/source/browse/wallet/src/de/schildbach/wallet/util/LinuxSecureRandom.java">his implementation</a> based directly on <code>/dev/urandom</code>.</p>
<p>As a user, you should check if there is an update available for any of your security-relevant applications, and when in doubt, stop using keys generated on one of the vulnerable Android versions (Android 4.2, and to a lesser degree also everything from before – the 64-bits of entropy again). I plan to evaluate some of the obvious candidates like ConnectBot myself.</p>
<p>This is my preliminary analysis on the situation - if you have anything to share on the matter, or have spotted some wrong conclusions in my argumentation, please leave your comment below or drop me a message!</p>
<p>A side note on the Bitcoin side of the ECC nonce problem: The original Bitcoin implementation elegantly avoids the problem, since it only reveals the public key at the moment of a transaction, and sends the "change" to a newly generated address, to which the ECC key remains unknown until the next transaction. This doesn't apply universally (for example if you receive transactions to an address which you have already used to send coins previously, e.g. well-known donation address; special transactions directly to public keys are also not protected). Since the method is based on a cryptographic hash of the public key, it even provides some protection against quantum-computing based attacks. I wonder what the exact motivation of the creators for using hashes was, but it sure is a nice trick! Unfortunately, bitcoinj currently sends all change to the original address, to which the key has already been revealed, which is why the attack worked in the first place. But there is a good reason to do this: Until we get deterministic address generation, ad-hoc creating addresses to transfer the change to makes wallet backups difficult and is prone to inadverted loss of private keys. But this is probably the topic of another blog post.</p>Uninitialized buffers in OpenGL2013-05-20T21:25:00+02:00lxgrtag:blog.lxgr.net,2013-05-20:posts/2013/05/20/uninitialized-buffers-in-opengl/<p>As I've mentioned in my last article, I'm interested in the implementation details and the security of open and closed-source GPU drivers.</p>
<p>In addition to the security implications of the model that is used by some of the current drivers (they allow the OpenGL client to send commands directly to the GPU, with the kernel only checking for illegal address references in the command stream, instead of using an actual IOMMU), there is a much simpler way to cause mischief when given access to an accelerated OpenGL implementation on a system: Uninitialized buffers.</p>
<p>Normally, when requesting memory from the operating system (for example through the malloc standard library function, which in turn uses an anonymous, private mmap memory mapping), the kernel goes through the effort of zeroing out the contents of the newly allocated chunk of main memory. While this is not required by the C language specification in any way, and one should never rely on that implementation detail (smaller allocations could be handled by the library in a different way, and those are not guaranteed to be zero-initialized), it's a pretty important <a href="http://stackoverflow.com/questions/1622196/malloc-zeroing-out-memory">security feature</a>.</p>
<p>Just imagine what would happen if the physical memory block used to be allocated to your browser, and contained the session cookie for an online banking session, or worse, an instance of GPG, containing your private key... And while most security-relevant code will probably go to great lengths to avoid that kind of thing from happening by overwriting the relevant memory locations before deallocating them, there is always the possibility of application crashes, which will render those protections useless.</p>
<p>All in all, that operating system feature is really essential to guarantee the isolation among different users who are working on the same machine simultaneously. (As a side note, <a href="http://lwn.net/Articles/322823/">a similar thing</a> is performed for some file systems, which don't zero out newly-allocated blocks, but use a different method to achieve a similar effect, and to prevent users from gaining access to residual chunks of data in case of a system crash during the allocation.)</p>
<p>I was expecting the same thing to happen for GPU driver implementations, since nowadays, many window managers use OpenGL acceleration to draw the window contents to the right locations with various effects like transparency or animated window switching. Basically, the window content is stored as an OpenGL texture, which is later mapped to a rectangle on the graphical desktop. So, in many cases, their content is at least as security-critical as the content of main memory – just think about your terminal's or browser's window content. Well, it turns out I was wrong:</p>
<p><img alt="An uninitialized OpenGL texture" src="//blog.lxgr.net/images/gl-buffer-res.png" /></p>
<p>This screenshot shows a simple OpenGL demonstration program, which I modified just a tiny bit: I removed the part that loads the cube texture from memory, or more accurately, replaced the pointer to the image data with a null pointer (which seems to be allowed by the OpenGL specification). It is implementation-defined whether that means that the buffer should be zero-initialized, or can remain uninitialized – and the <a href="http://nouveau.freedesktop.org/wiki/">nouveau</a> driver for my Nvidia card seems to do the latter, apparently for performance reasons.</p>
<p>I asked the nouveau developers in the IRC channel for their view on the topic, and Dave Airlie told me that while video buffers in the main memory should be zero-initialized on nouveau, buffers residing in video memory are not overwritten by default, while theoretically possible.</p>
<p>On integrated GPUs that use the main memory for all of their buffers, the problem could be even more severe – not only the content of other user's windows, but even arbitrary memory contents could be theoretically extracted with custom shader code. I retried the experiment on an Intel GPU, and was relieved to only see an untexturized black cube. The same thing happens on Android, where I tried it on both an Adreno- and an Nvidia Tegra–equipped device. However, this does not mean that those platforms are safe – it only means that somewhere in their OpenGL implementation, the buffer is zeroed, which might as well happen only in the userspace library, and could therefore be circumvented by directly interfacing with the command buffer (which is admittedly much more difficult, and might well be impossible for things like WebGL, where direct access to those buffers is not possible for application code).</p>
<p>One possible mitigation for that security risk is very simple, and therefore widely used: Just don't give access to the video hardware to anyone but users that are physically present. Many Linux distributions do just that with the <code>allowed_users=console</code> setting of the <code>Xwrapper.config</code> configuration file. This reduces the attack surface significantly – most computers are only used for desktop logins by a single person at a time, and anybody who is able to run software in that user's X session (which seems to be an <a href="http://dri.freedesktop.org/wiki/DRM/">additional requirement</a> for GPU hardware access, at least on DRI/DRM) has much easier ways to grab arbitrary window contents.</p>
<p>But with WebGL becoming more and more popular, that situation is changing – now, web page authors can execute OpenGL code on any visitor's GPU hardware, and read back the content of the resulting images (with limits imposed by the same-origin policy). This might be one of the reasons why WebGL <a href="http://www.khronos.org/webgl/security/#Access_to_Uninitialized_Memory">specifically mandates</a> that implementations clear their buffers before allocation. That's obviously a very good idea, seeing that there is even a <a href="http://www.contextis.com/research/blog/webgl-more-webgl-security-flaws/">working exploit</a> for that particular loophole! Now let's hope that all browser vendors read that part of the specification carefully, and we should be safe – but only against that specific security threat of running untrusted code on hardware with direct access to the main memory...</p>Graphics processing in hardware and software2013-05-10T20:23:00+02:00lxgrtag:blog.lxgr.net,2013-05-10:posts/2013/05/10/graphics-processing-hardware-software/<p>I've got a peculiar hobby: I like to worry about very specific implementation details of technologies I don't really understand at all; one of them being GPUs and graphics drivers.</p>
<p>On one hand, it's really simple: In almost every computing device, there is a GPU. This is basically a programmable, special-purpose, massively parallel CPU, and until recently, its only purpose was drawing triangles in different colors; and not just one or two, but lots of them – per second. Parts of it are dedicated to the triangle-drawing business, because that's still the most efficient way to do it, but most of the hard work happens in the programmable parts.</p>
<p>Since every device seems to need a driver, there is one for every GPU. And how hard can <em>that</em> be? Identify the triangle-drawing chip in question, figure out a way to talk to it, throw some triangle coordinates at it and marvel at the results.</p>
<p>But the more I think and read about those two components, the more I get the impression that it might not be that simple.</p>
<p>Concerning the GPU itself, I'm wondering what parts of the rendering pipeline (the process of interpreting large amounts of bits as triangle coordinates and textures and converting them to a rasterized 2D projection of a three-dimensional scene) are actually still happening in dedicated circuits, and how much of it really happens on general purpose CPUs, programmed by firmware internal to the GPU or possibly even the driver, and therefore the CPU. From what I've learned so far (mostly by reading lots of introductions to <a href="http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Table-of-Contents.html">OpenGL</a>, <a href="http://fgiesen.wordpress.com/2011/07/09/a-trip-through-the-graphics-pipeline-2011-index/">modern GPUs</a>, <a href="https://01.org/linuxgraphics/documentation/driver-documentation-prms">technical documentations</a> and <a href="http://kernel.org/">source code</a>, everything is possible – there are software renderers that run as software on the shaders of a GPU, and, on the other end of the spectrum, "hardware" components that are fed with ASCII representations of OpenGL shaders (with the help of <a href="http://www.phoronix.com/scan.php?page=news_item&px=MTIxNDk">not-so-open source drivers</a>.</p>
<p>Some GPUs need <a href="http://phoronix.com/forums/showthread.php?38445-Confused-by-firmware">blobs of firmware</a> in order to do their job (which hints to a partial software-like approach to the problem); others <a href="http://phoronix.com/forums/showthread.php?79473-Digging-Deeper-Into-AMD-s-UVD-Code-Drop&p=323934#post323934">don't</a> – but that doesn't say anything, since firmware can also be stored inside of a chip, similar to the microcode of common "CISC" CPUs.</p>
<p>The more I think about that, the more I realize that, for this topic as for almost every other technical subject, there is no easy or general answer, and finding it the hard way takes lots of time, and also luck with finding the right documentation. Which brings me to the topic open source graphics drivers.</p>
<p>Since most of the magic seems to be happening at least partially in software, whether on the host CPU or in embedded DSPs of the GPU (though I realize that there are quite a few ASICs left), there is an understandable, but still annoying tendency of GPU vendors to treat their driver software with as much secrecy as their actual hardware products – simply because that actual product is actually the combination of the chip and the driver.</p>
<p>This brings us the obvious problems that all closed-source drivers share: We have no way of fixing problems when they arise, and also no way of making assertions, or even educated guesses, about the security properties of a software that runs with the highest privileges possible on millions, possibly billions, of machines storing sensitive data, both commercial and private.</p>
<p>Apart from actual vulnerabilities in the driver code running on the CPU, I'm <a href="http://security.stackexchange.com/questions/35634/is-opengl-a-security-problem">wondering</a> to what extent processes running on the GPU itself can access the main memory of the system, and how the various drivers ensure that such memory accesses don't circumvent the process separation that is now commonplace on most operating systems thanks to the memory virtualization provided by the combination of the memory management unit of the CPU and the security mechanisms of the operating system kernel.</p>
<p>Since shaders, the programs running on the GPU execution units, can be provided in source and sometimes also binary form by any user of the graphics (OpenGL) or general purpose (OpenCL) API, memory accesses of those shaders have to be obviously limited to something less than the whole system memory space. There seem to be two approaches:</p>
<ul>
<li>For some GPU drivers, that protection is provided by the driver verifying all commands that are submitted from the user space to the GPU. It checks for illegal memory accesses and other potentially dangerous operations.</li>
<li>Other, mostly newer models provide a hardware MMU themselves that can be programmed by the operating system or the driver to disallow all memory accesses, except for the ones for data that is located in buffers owned by the same user.</li>
</ul>
<p>According to a <a href="http://phd.mupuf.org/files/fosdem2013_drinext_drm2.pdf">presentation on the subject</a>, the first approach is currently used by the Linux drivers for AMD and Intel GPUs, while the second one seems to be only supported by the open nouveau driver for Nvidia GPUs.</p>
<p>The situation for OpenGL on Android seems different, even though it also uses the Linux kernel: Due to <a href="https://android.googlesource.com/kernel/tegra/+/android-tegra3-grouper-3.1-jb-mr1-fr/drivers/gpu/ion/">some references</a> in the Kernel source code of almost all Android platforms which I examined, I suspect that most or all of the Android drivers actually use an IOMMU, that is, the hardware approach to the problem. I suspect that this is because it allow the mobile GPU vendors to open-source the Kernel portion of their drivers – the verification approach can obviously only be executed in the Kernel (or a trusted userspace daemon, with even more overhead), and needs a lot of knowledge about the format of the command stream sent to the GPU, which would thereby be openly documented.</p>
<p>As I've mentioned, most of the drivers are released as closed-source by their vendors (with Intel and possibly (I've not done any research on them) AMD being a laudable exception), but there are some open-source alternatives, most of them are created by tediously <a href="http://blog.emmanueldeloget.com/index.php?post/2013/03/08/The-SoC-GPU-driver-interview">reverse-engineering the GPUs</a>. At least for Nvidias Tegra line of mobile GPUs, that might change, though; after <a href="http://www.wired.com/wiredenterprise/2012/06/torvalds-nvidia-linux/">fingers having been pointed</a> at each other, Nvidia finally seems to <a href="http://www.phoronix.com/scan.php?page=news_item&px=MTE5MTc">release a bit more</a> to the open source community in the form of both documentation and actual code commits. <a href="http://lwn.net/Articles/467769/">One of them</a> is especially interesting to me, since it confirms the IOMMU approach being used. On the mainline Linux kernel, it also <a href="https://gitorious.org/linux-tegra-drm/pages/Host1xIntroduction#Stream+validation">seems possible</a> to use the stream validation approach.</p>
<p>So what is my point? As I've said, I have a peculiar hobby, and somehow I find the topic of GPU drivers really interesting. I still don't know nearly enough even to be able to understand the Kernel source code, but I'll continue to try to get a clearer overview nevertheless. If you've got any hints for me, please go ahead and write me (blog at lxgr dot net)!</p>A quine in x86-64 assembly2013-05-10T17:30:00+02:00lxgrtag:blog.lxgr.net,2013-05-10:posts/2013/05/10/a-quine-in-x86-64-assembly/<p>This summer term, I'm taking a really interesting course on computer security: While the lectures are pretty theoretical (one of the topics is a proof that shows that proving the general security properties of certain models is equivalent to the halting problem, which is done by implementing a turing machine within the access model...), the homeworks are partially about x86(-64) assembly programming. My only assembly programming experiences until now were with MMIX, which is almost completely on the opposite end of the RISC/CISC spectrum than the good old x86 (not to mention that the architecture is completely theoretical and has never been implemented in hardware). To make a long story short, I finally have a reason to program some x86 assembly!</p>
<p>Our most recent exercise sounds quite simple, but kept me busy longer than I expected: We are supposed to write a quine in x86-64 assembly, that is, a program that has its own source code as its (only) output – identical to the last byte. I was already familiar with quines, but I had never tried to create one before, and if I had, assembly would definitely not have been my language of choice, but since it was one of two non-optional homework exercises, I figured that it couldn't be so hard after all. So without further ado, here is a quine in x86-64 assembly (GNU assembler syntax):</p>
<div class="highlight"><pre><span class="na">.att_syntax</span> <span class="no">noprefix</span>
<span class="na">.globl</span> <span class="no">main</span>
<span class="nl">main:</span>
<span class="nf">pushq</span> <span class="no">rbp</span>
<span class="nf">movq</span> <span class="no">rsp</span><span class="p">,</span> <span class="no">rbp</span>
<span class="nf">mov</span> <span class="no">$.Cs</span><span class="p">,</span> <span class="no">rdi</span>
<span class="nf">mov</span> <span class="no">$0xa</span><span class="p">,</span> <span class="no">rsi</span>
<span class="nf">mov</span> <span class="no">$0x22</span><span class="p">,</span> <span class="no">edx</span>
<span class="nf">mov</span> <span class="no">$.Cs</span><span class="p">,</span> <span class="no">ecx</span>
<span class="nf">mov</span> <span class="no">$0x22</span><span class="p">,</span> <span class="no">r8d</span>
<span class="nf">mov</span> <span class="no">$0xa</span><span class="p">,</span> <span class="no">r9d</span>
<span class="nf">xor</span> <span class="no">eax</span><span class="p">,</span> <span class="no">eax</span>
<span class="nf">call</span> <span class="no">printf</span>
<span class="nf">xor</span> <span class="no">eax</span><span class="p">,</span> <span class="no">eax</span>
<span class="nf">leave</span>
<span class="nf">ret</span>
<span class="nl">.Cs:</span> <span class="na">.string</span> <span class="s">".att_syntax noprefix</span>
<span class="s">.globl main</span>
<span class="s">main:</span>
<span class="s">pushq rbp</span>
<span class="s">movq rsp, rbp</span>
<span class="s">mov $.Cs, rdi</span>
<span class="s">mov $0xa, rsi</span>
<span class="s">mov $0x22, edx</span>
<span class="s">mov $.Cs, ecx</span>
<span class="s">mov $0x22, r8d</span>
<span class="s">mov $0xa, r9d</span>
<span class="s">xor eax, eax</span>
<span class="s">call printf</span>
<span class="s">xor eax, eax</span>
<span class="s">leave</span>
<span class="s">ret%c.Cs: .string %c%s%c%c"</span>
</pre></div>
<p>It can be compiled, executed and verified for proper quine-ness like this:</p>
<div class="highlight"><pre>gcc quine.s -o quine <span class="o">&&</span> ./quine > output <span class="o">&&</span> diff quine.s output
</pre></div>
<p>As do most quines, this probably needs some explanation. Generally speaking, all quines (at least the ones I've come across) share a common structure: There is some code of the language in question, and one or more rather long strings, which contain most of that code in quoted form. The trick is to print the quoted code twice: Once verbatim, and once with quotation marks and some additional characters, so that the string declaration itself is printed out. To do that, the string quotation signs have to be escaped or stored in some modified form – otherwise, they would be interpreted simply as the end of the string.</p>
<p>The most obvious guess is to just escape them with a backslash (like so: <code>\"</code>), but that doesn't really help us – now we also have to print a backslash in our output! The solution here (and in many other quines) is to store them in another form that can be safely quoted verbatim inside of a string, but still evaluates to the desired character in the output. In this case, the <code>printf</code> C library function is used to recreate two occurences of two otherwise problematic characters: The aforementioned quotation mark and the newline character. Both characters are stored as their numeric (or more specific, hexadecimal) representation according to the ASCII codepage: <code>0xa</code> for the newline, and <code>0x22</code> for the double quotation mark. All that I was able to learn from the very nice example for a quine in C given in the (German) <a href="http://de.wikipedia.org/wiki/Quine_(Computerprogramm)">Wikipedia article on the subject</a>, which also taught me the really neat trick of using the same string twice – once as a formatting string, and once again as one of the parameters of the <code>printf</code> function.</p>
<p>My approach was then to find a valid translation for that quine to assembly, which revolved mostly around two problems: Generating the machine instructions for the <code>printf</code> call, and escaping all occurences of problematic characters in the resulting program so that it can be stored as a valid string.</p>
<p>The first part of the problem can be easily solved by some experimentation with a similar C program and by disassembling the binary as compiled by GCC (and identifying the relevant lines in the output!) – it boils down to moving the address of the string and the literal values of the ASCII characters to the right registers (according to the x86-64 calling convention) and executing the call.</p>
<p>A minor detail of interest is the instruction <code>xor eax, eax</code> right before the function call: As it turns out, functions with variable-length parameter lists like <code>printf</code> expect the number of parameters passed to them in the <code>eax</code> register; in this case, there are exactly zero. I can only speculate about the reasons for this part of the calling conventions (after all, the total number of arguments is <em>not</em> passed in a register!), but I gather it has something to do with possible optimizations in functions further down/up the call stack, since saving those registers is rather costly and should be avoided if not necessary. I only figured out the importance of zeroing the register when I tried the program on a workstation at my university – while I could get away without it on my own laptop, it would invariably crash there without that instruction.</p>
<p>Another problem would have been the newline characters: Since the GASM syntax requires a newline after every machine instruction statement, it's not possible to get away like the C quine from the mentioned article, that is, to simply write the quine on a single line. Fortunately, GCC/GASM does the right thing when confronted with "raw" newline characters inside of a string, and just treats them the same way it would handle a proper <code>\n</code> newline. This causes some warnings from my version of GCC, but compiles/assembles nevertheless – otherwise, all the newline characters would probably have to be submitted as parameters for <code>printf</code>.</p>
<p>If you are familiar with the GASM assembly syntax, you might have noticed a minor oddity about the code: Register names are not prefixed with a percent sign as they usually have to be. The reason for that is that the percent sign has a very special meaning in the formatting string parameter of <code>printf</code> – it indicates that the following character(s) should be interpreted as a formatting directive, and replaced by a specific parameter of the function! This leads to a problem similar to simple backslash escaping: For every percent character, an ASCII-encoded percent sign has to be given to <code>printf</code> as a parameter, but for every new parameter, we need a new <code>mov</code> instruction to a register – which includes a percent sign...</p>
<p>This part of the problem is <a href="http://codegolf.stackexchange.com/a/609">actually easier to solve on x86</a> (the 32 bit variant): Since all function parameters are there passed on the stack, they can be pushed there with the <code>push</code> instruction (<code>push $0x22</code>, <code>push $0x0a</code>, ...) – no percent sign necessary! On x86-64, the first 13 parameters are passed in processor register instead, which means that additional parameters would have to be generated by using up the first few parameter slots in a way that still creates the same output – not impossible, but very tedious (both in manual execution and code size/readability).</p>
<p>A trick to circumvent that problem is the use of the <code>.noprefix</code> directive of GCC/GASM: Since a percent sign in front of a variable is only a visual aid to the programmer and not necessary to correctly parse the program, this option allows us to simply omit all the percent prefixes – which is just what we need.</p>
<p>After the encoding has been taken care of, all that is left is the exact structure of the <code>printf</code> format string. As I've mentioned, every occurence of a percent character is replaced by one of the function parameters in the output, and by careful construction of the formatting string, together with the trick of using the very same string as both a formatting specification for printf and a parameter being substituted inside that formatting string it is possible to create an output that is exactly identical to the source code – a quine!</p>Jumboframes on the Internet?2013-04-12T11:43:00+02:00lxgrtag:blog.lxgr.net,2013-04-12:posts/2013/04/12/jumboframes-on-the-internet/<p>Recently, I've been experimenting with Wireshark for my bachelor's thesis, monitoring the performance of TCP uploads from my notebook to my web server. A while ago, I had also swapped my router for a nicer model capable of gigabit ethernet and 5 GHz wifi (due to increasing congestion of the 2.4 GHz band in my apartment building), of course also running OpenWrt <a href="//blog.lxgr.net/posts/2013/01/28/my-openwrt-setup/">like the old one</a>.</p>
<p>Soon after the switch, I noticed an oddity in the Wireshark captures: Some of the outgoing TCP segments were reported as having a length of more than 1500 bytes, which is the upper payload limit for Ethernet, and therefore also for most, if not all, residential Internet connections.</p>
<p><img alt="Ethernet frames bigger than 1500 bytes" src="//blog.lxgr.net/images/wireshark_tcp_gso.png" /></p>
<p>Since I didn't really expect the path between my notebook and my server to be capable of a MTU higher than 1500, the obvious explanation for this to work would be IP fragmentation occuring in my router, which would be very unfortunate to say the least.</p>
<p>To figure out if that was really the case, I started tcpdump on the server, with the following interesting result:</p>
<p><img alt="Ethernet frames containing TCP segments, as received by the server" src="//blog.lxgr.net/images/wireshark_tcp_normal.png" /></p>
<p>No sign of IP fragmentation whatsoever; the packets were arriving as if sent with an MTU of 1500! This also matches both the MTU configured on my notebook and explains the pattern of ACKs received from the server – usually, every other segment should be ACKed by the recipient, but in this case, I was receiving much more than that.</p>
<p>After a bit of googling, I found the explanation: <a href="http://en.wikipedia.org/wiki/Large_segment_offload">TCP segementation offload (TSO)</a>. This is (theoretically) a nice feature of some NICs that allow the operating system to delegate TCP segmentation and TCP and IP header generation to the network interface, relieving the CPU of those duties and possibly also increasing performance quite a bit. However, if there are bugs in the NIC firmware, this could lead to very obscure and hard to debug transmission errors, and it also makes debugging other network behavior more difficult, as I had experienced myself with my measurements.</p>
<p>There is an easy way to disable TSO on Linux:</p>
<div class="highlight"><pre>sudo ethtool -K eth0 tso off
sudo ethtool -K eth0 gso off
</pre></div>
<p>GSO is a very similar technology, which can be used to offload some of the higher-layer networking tasks from the kernel to the network interface for protocols other than TCP. It also has to be disabled, because according to my tests, it happily takes over TSO's duties once disabled, also causing strage results in Wireshark.</p>
<p>Apart from the confusion in my packet traces, I have yet to experience any side effects of TSO or GSO with my NIC (likely because of the driver which is commendably developed by the NIC vendor itself), and plan to leave them enabled while I'm not working with Wireshark.</p>VPNs and IPv6, part 22013-03-29T17:28:00+01:00lxgrtag:blog.lxgr.net,2013-03-29:posts/2013/03/29/vpn-ipv6-part2/<p>As I've <a href="//blog.lxgr.net/posts/2013/03/06/vpn-circumvention-ipv6/">written before</a>, VPNs can lead to insecure situations when used with IPv6 enabled networks.</p>
<p>The easiest way to mitigate that problem is actually just to enable IPv6 tunneling over the VPN itself, provided your VPN gateway has IPv6 connectivity and you have a spare /64 subnet you can dedicate to the VPN clients. (Unfortunately, this is the smallest subnet size OpenVpn is willing to accept). My provider has agreed to make an appropriate subnet available to my server, but I haven't been able to try it so far.</p>
<p>If that's not possible for you, e.g. due to IPv6 being unavailable at your VPN gateway, there is a simple workaround that breaks IPv6 connectivity for all connected clients: Just hand out bogus IPv6 addresses and routes to all clients, and drop all IPv6 traffic on the server. This is of course not as nice as an option to cleanly disable IPv6 connectivity, but at least for the Android client, I'm not aware of any other solution so far.</p>
<p>The following two lines in the OpenVPN server.conf should do the trick:</p>
<div class="highlight"><pre>server-ipv6 ::1/64
tun-ipv6
</pre></div>
<p>Make sure to disable IPv6 forwarding on the VPN server to avoid any surprises (e.g. link-local IPv6 connectivity to other servers on the same subnet):</p>
<div class="highlight"><pre>sysctl net.ipv6.conf.all.forwarding=0
</pre></div>
<p>Try the setup by connecting to the VPN and accessing one of the innumerable "what-is-my-IPv6"–services from your client to make sure it works as expected.</p>Static blogs and HTTP caching2013-03-15T10:56:00+01:00lxgrtag:blog.lxgr.net,2013-03-15:posts/2013/03/15/static-blog-http-caching/<p>As you can see in the footer, this blog is powered by <a href="http://getpelican.com">Pelican</a>, a static blog generator written in Python. It's really simple to use and fits my requirements nicely – I can write posts offline on my notebook and view the results in my browser with the included web server, it doesn't require any insecure server-side software (the output is plain HTML, CSS and a bit of JavaScript for browsers that are not quite up to date) and is very easy on server resources because by default, almost everything can be cached by web browsers.</p>
<p>However, there is one annoying side effect of everything being cached: Since that also includes the landing page, new posts could be invisible to recurring visitors for quite a while. In a bit more detail, here is what is going on at the HTTP level:</p>
<p>By default, my webserver, lighttpd, delivers all static HTML pages with no explicit caching headers, but includes the modification time of the resource (only the relevant headers are included) and an <a href="http://en.wikipedia.org/wiki/HTTP_ETag"><code>ETag</code></a>:</p>
<div class="highlight"><pre>Date: Fri, 15 Mar 2013 10:03:43 GMT
ETag: "4531062"
Last-Modified: Thu, 14 Mar 2013 20:12:06 GMT
</pre></div>
<p>The <code>ETag</code> is good to have (browsers can use it to unambiguously revalidate cached content with the server, as I'll explain later), but the <code>Last-Modified</code>–Header combined with <em>no</em> explicit statement about cacheability triggers a heuristic <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.2.4">defined in HTTP</a> in most browsers. Basically, browsers calculate the difference between the time the resource was retrieved and the time it was last modified on the server, and cache the resource for 10% of that value <em>without revalidating with the server</em>.</p>
<p>This means that for a blog that is daily updated with new posts, users will eventually see the posts after a few hours after their last visit, but for a blog that hasn't been updated for several weeks or months, ten percent of that time can be pretty significant.</p>
<p>A simple solution is to just manually define a cache validity in the HTTP headers for some or all resources. lighttpd has the <a href="http://redmine.lighttpd.net/projects/1/wiki/Docs_ModExpire"><code>expires</code> module</a> that does just that. Here is the relevant line in my <code>lighttpd.conf</code>:</p>
<div class="highlight"><pre><span class="k">expire.url</span> <span class="p">=</span> <span class="s">(</span> <span class="s">"/theme/"</span> <span class="p">=</span><span class="s">></span> <span class="s">"access</span> <span class="s">plus</span> <span class="mi">7</span> <span class="s">days",</span> <span class="s">""</span> <span class="p">=</span><span class="s">></span> <span class="s">"access</span> <span class="s">plus</span> <span class="mi">1</span> <span class="s">hours"</span> <span class="s">)</span>
</pre></div>
<p>The effect is that all resources in the subdirectory <code>theme</code> will have an <code>Expires</code> header 7 days in the future, and everything else will be valid for just an hour. This is a tradeoff between server and client resource usage and immediate updates: For me, an hour of delay is not a big deal, and users jumping back and forth between blog posts will be able to do so without any further HTTP requests. Here are the response headers of the main blog page:</p>
<div class="highlight"><pre>Cache-Control: max-age=3600
Date: Fri, 15 Mar 2013 10:23:06 GMT
ETag:"4531062"
Expires: Fri, 15 Mar 2013 11:23:06 GMT
Last-Modified: Thu, 14 Mar 2013 20:12:06 GMT
</pre></div>
<p>As you can see, the <code>max-age</code> directive exlicitly states a validity of 3600 seconds, and the <code>Expires</code> header also points to a value one hour in the future.</p>
<p>Even when that time is reached, the whole resource doesn't have to be transferred again: Browsers can just perform a conditional HTTP request using the <code>ETag</code> or <code>Last-Modified</code> headers that they cache together with the resource itself. If the content is still the same, the server will be able to deduce that from the headers and reply with a <code>304 Not Modified</code> HTTP response. As long as your site is not very highly frequented or references many additional resources, cache revalidation is not too expensive.</p>
<p>One thing that has also helped me tremendously in understanding HTTP caching was <a href="http://stackoverflow.com/a/385491/1016939">an answer on Stackoverflow</a> that explains how to force the various browsers to revalidate a resource or to completely bypass the cache – for debugging, it's very useful to know that there is a big difference between pressing <code>F5</code> or <code>Ctrl + F5</code> in most browsers.</p>Variable indirection in shell scripts2013-03-13T15:30:00+01:00lxgrtag:blog.lxgr.net,2013-03-13:posts/2013/03/13/variable-indirection-shell-scripts/<p>Recently, I had to find a way to do variable indirection in a shell script. More specifically, I wanted to write a function that takes two arguments and interprets one of them as a string, and the other one as a variable to which that string should be added – a simple append function.</p>
<p>Usually, that would be a good occasion to switch to some more comfortable scripting language than the unix shell, but sometimes that's not possible. So here is how to do it (thanks to <a href="http://tldp.org/LDP/abs/html/ivr.html">an article on TLDP</a>):</p>
<div class="highlight"><pre>append<span class="o">()</span> <span class="o">{</span>
<span class="c"># Appends the value of $1 to the variable indicated by $2</span>
<span class="nb">eval</span> <span class="nv">$2</span><span class="o">=</span><span class="se">\"\$</span><span class="nv">$2</span> <span class="nv">$1</span><span class="se">\"</span>
<span class="o">}</span>
</pre></div>
<p><a href="http://tldp.org/LDP/abs/html/internal.html#EVALREF"><code>eval</code></a> is a very useful shell built-in that converts a string to a command, performing the regular shell variable substitution. In the small function above, this means that when calling the function like this:</p>
<div class="highlight"><pre><span class="nv">A_VARIABLE</span><span class="o">=</span><span class="s2">"initial value"</span>
append <span class="s2">"some string"</span> <span class="s2">"A_VARIABLE"</span>
</pre></div>
<p>The line</p>
<div class="highlight"><pre><span class="nb">eval</span> <span class="nv">$2</span><span class="o">=</span><span class="se">\"\$</span><span class="nv">$2</span> <span class="nv">$1</span><span class="se">\"</span>
</pre></div>
<p>first becomes (by regular shell variable substitution)</p>
<div class="highlight"><pre><span class="nb">eval </span><span class="nv">A_VARIABLE</span><span class="o">=</span><span class="s2">"</span><span class="nv">$A_VARIABLE</span><span class="s2"> some string"</span>
</pre></div>
<p>which is then evaluated as a command, again with variable substitution:</p>
<div class="highlight"><pre><span class="nv">A_VARIABLE</span><span class="o">=</span><span class="s2">"initial value some string"</span>
</pre></div>
<p>At least, I hope that this is what is actually going on... Quote escaping in shell scripts can be tricky sometimes. Many more useful examples of indirect references can be found in the referenced article.</p>TLS and RC4 - not so secure after all2013-03-13T11:24:00+01:00lxgrtag:blog.lxgr.net,2013-03-13:posts/2013/03/13/tls-rc4-not-so-secure/<p>Turns out that TLS with RC4 (which was <a href="http://blog.phonefactor.com/2011/09/23/slaying-beast-mitigating-the-latest-ssltls-vulnerability/">supposed to protect us</a> against the BEAST and the CRIME attacks) is <a href="http://www.isg.rhul.ac.uk/tls/">not so secure after all</a>:</p>
<blockquote>
<p>The attacks arise from statistical flaws in the keystream generated by the RC4 algorithm which become apparent in TLS ciphertexts when the same plaintext is repeatedly encrypted at a fixed location across many TLS sessions.</p>
</blockquote>
<p>That sounds familiar... A few months ago, I read a very similar statement in a paragraph on attacks on RC4 <a href="http://tools.ietf.org/html/rfc4345#section-5">in RFC4345</a> (Improved Arcfour Modes for SSH):</p>
<blockquote>
<p>[...] A consequence of this is that encrypting the same data (for instance,a password) sufficiently many times in separate Arcfour keystreams can be sufficient to leak information about it to an adversary.</p>
</blockquote>
<p>Intrigued by that, I posted <a href="http://crypto.stackexchange.com/questions/3451/is-rc4-a-problem-for-password-based-authentication/">a question</a> on Stackexchange Cryptography, asking if the same problem wouldn't also apply to TLS, with pretty bad implications for password/cookie authentication. I got a very interesting response by a user named poncho, who claimed that he was able to successfully recover a password from 8 billion RC4 encrypted messages.</p>
<p>8 billion seems like too much for a practical attack even when the attacker is able to provoke repated retransmissions of the secret, but if there were a way to optimize that attack, TLS with RC4 would be in serious trouble. And this seems to be exactly what happened just now.</p>
<p>Matthew Green has published a very nice <a href="http://blog.cryptographyengineering.com/2013/03/attack-of-week-rc4-is-kind-of-broken-in.html">summary</a> of the new attack and the implications on his blog, and I completely agree with his conclusion – we need to stop using RC4.</p>Server relocation2013-03-08T11:46:00+01:00lxgrtag:blog.lxgr.net,2013-03-08:posts/2013/03/08/server-move-graz-vienna/<p>This weekend, the server on which this blog is hosted will be <a href="http://www.edis.at/de/support-und-service/blog/edis-zieht-um-nach-wien/">moved from Graz to Vienna</a>. If all goes well, there will be a short outage on Saturday evening/night, and much better connectivity afterwards.</p>VPNs and IPv62013-03-06T10:31:00+01:00lxgrtag:blog.lxgr.net,2013-03-06:posts/2013/03/06/vpn-circumvention-ipv6/<!--- Summary: Inadverted VPN circumvention by IPv6 -->
<p>A while ago, I have configured a small OpenVPN for personal use (mostly for security when using public wireless networks) with <a href="http://openvpn.net/">OpenVPN</a>. The setup is pretty easy, thanks to a <a href="http://wiki.openvpn.eu/index.php/Konfiguration_eines_Internetgateways">very helpful tutorial (in German)</a> and the sensible default settings of OpenVPN itself. (Setting up the certificate infrastructure was a bit annoying, though – I would really prefer an SSH-like approach where the users can create a private key for themselves, and the VPN server has a list of the key/user mappings, but that's another story.)</p>
<p>Configuring the server to push a default route to the clients is as simple as setting the <code>push redirect-gateway def1</code> option in the server configuration, and mostly, that works as expected.</p>
<p>However, there is a huge caveat for IPv4-only clients. Since I don't have an IPv6 subnet big enough to provide IPv6 tunneling on my server as well (OpenVPN, or at least the version included in Ubuntu 12.04, seems to require a /64 subnet for now, but my provider only provides a tiny /112), I just didn't configure IPv6 and expected IPv6 connectivity to be broken. But that's not what's happening:</p>
<p>When connecting to the VPN from a client that has both IPv4 and IPv6 connectivity, only the IPv4 traffic will be routed over the VPN gateway, <em>but the IPv6 traffic will be routed locally</em> – and since the <a href="http://www.worldipv6launch.org/">world IPv6 launch</a>, there are quite a lot of hosts that are reachable over IPv6 and serve AAAA records to all users. Except for TLS protected connections, traffic to them will be unencrypted, and even then, the connection metadata (IP addresses etc.) will be plainly visible to anybody on the public network.</p>
<p>After thinking about that for a while, it kind of makes sense: At no point did I instruct OpenVPN to <em>break</em> my existing IPv6 connectivity, and since I didn't provide any IPv6 tunneling settings, my routes for that were just left alone. It can also be fixed easily enough – just configure a script that tears down IPv6 connectivity before connecting to the VPN, and restores it immediately after that. Maybe there is even a way to do that from the server via <code>push</code> instructions, but I've had no success with that so far.</p>
<p>Curious about the issue, I decided to check how other VPN solutions and clients handle that situation, with pretty much the same results:</p>
<p>Cisco's AnyConnect, when used with OpenConnect, behaves exactly like OpenVPN. The Android client, however, seems to specifically work around that problem – IPv6 connectivity breaks while connected to an IPv4-only VPN. I haven't been able to find out how that works, but I suspect that either some additional routes are pushed to the client, and the IPv6 traffic is discarded locally or at the gateway, or something is going on at the DNS level (since I get <code>NAME_NOT_RESOLVED</code> errors when visiting <em>what-is-my-IPv6</em>–like sites from the VPN).</p>
<p>OpenVPN for Android behaves just like the Linux client. Unfortunately, unlike Linux, Android provides no way for the user or the VPN application developer to disable IPv6, which makes a workaround pretty much impossible. I've reported that as <a href="https://code.google.com/p/ics-openvpn/issues/detail?id=142">a bug</a> to the developer, even though it is a problem with android, not OpenVPN – maybe he'll figure out a solution. (I've also reported it as <a href="https://code.google.com/p/android/issues/detail?id=48417">an Android bug</a>, but I'm really not sure if there's anybody from Google watching that bug tracker...)</p>
<p>I'm not really sure what would be the best solution to the problem: Should VPN clients just break IPv6 connectivity by default, to protect the data of users who will most likely be using a VPN under the assumption that it will be doing just that? Or should it leave IPv6 alone by default, just providing an option to automatically disable their IPv6 connectivity when needed? I'm in favor of the first approach, but as it is, even the second one would be a big step forward – on Android, users have <a href="https://code.google.com/p/android/issues/detail?id=32216"><em>no</em> way of disabling IPv6 traffic circumventing their VPN connection</a>, and most will not even know that it's happening.</p>My OpenWrt setup2013-01-28T12:15:00+01:00lxgrtag:blog.lxgr.net,2013-01-28:posts/2013/01/28/my-openwrt-setup/<!--- Summary: My OpenWrt configuration -->
<p>This weekend, I finally reinstalled <a href="https://openwrt.org/">OpenWrt</a> on my home router. I've been using a nightly build for several months now, and it had been working just fine, but unfortunately, the opkg (OpenWrt's package manager) repositories for the nightly builds are updated every few days, and all of the kernel modules have hard dependencies on a specific kernel version. So in order to install a new kernel module, I would have to upgrade my OpenWrt version every time – not very convenient.</p>
<p>Luckily, there is a release candidate for the newest version, called "Attitude Adjustment", which is what I upgraded to. This went without any major problems and I was even able to keep my configuration, but I had to reinstall all packages not included by default. At this opportunity, I decided to document my configuration and installed packages.</p>
<h1>Avoiding Bufferbloat</h1>
<p><a href="http://www.bufferbloat.net/">Bufferbloat</a> is a nasty phenomenon that occurs mostly on residential internet connections. In a nutshell, hugely oversized buffers combined with static queue management in cable and DSL modems cause huge delays during large uploads, which can lead to a very bad experience for interactive applications. I use a SIP phone to make voice calls, and previously to my solution, it was nearly impossible to make a phone call at the same time as somebody uploading pictures or videos on the same network – now it works just fine.</p>
<p>The solution to Bufferbloat is two-fold: First, the available bandwidth has to be limited on the router to just below the bandwith actually available to the modem, in order to avoid any buffering in the modem. Then, some adaptive or even "fair" queueing algorithm like <a href="http://queue.acm.org/detail.cfm?id=2209336">CoDel</a> can be used. Both tasks can be achieved via the Linux packet scheduler and its managment tool, <code>tc</code>. Both are available as OpenWrt packages:</p>
<div class="highlight"><pre>opkg install tc kmod-sched
</pre></div>
<p>uci, OpenWrt's configuration system, doesn't support custom packet schedulers yet, so my setup script is implemented as a shell script that is executed once at every boot of the router.</p>
<table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23</pre></div></td><td class="code"><div class="highlight"><pre><span class="c">#!/bin/sh</span>
<span class="c"># Insert the necessary kernel modules</span>
insmod sch_htb
insmod sch_fq_codel
<span class="c"># Reset the queueing disciplines</span>
tc qdisc del dev eth1 root
tc qdisc del dev br-lan root
<span class="c"># Add a HTB queue to the internal interface (to limit upload speeds)</span>
tc qdisc add dev eth1 root handle 1: htb default 1
<span class="c"># Limit the upload speed to 2048 kbit/s (adjust this to just below your actual upload speed!)</span>
tc class add dev eth1 parent 1: classid 1:1 htb rate 2048kbit
<span class="c"># Enable CoDel as a queueing algorithm for the queue</span>
tc qdisc add dev eth1 parent 1:1 handle 11: fq_codel
<span class="c"># The same for download speeds - adjust accordingly</span>
tc qdisc add dev br-lan root handle 1: htb default 1
tc class add dev br-lan parent 1: classid 1:1 htb rate 32768kbit
tc qdisc add dev br-lan parent 1:1 handle 11: fq_codel
<span class="nb">exit </span>0
</pre></div>
</td></tr></table>
<p>This solved the Bufferbloat issue completely for me, but if it works for you depends on a lot of factors - test the script before enabling it at every boot.</p>
<h1>IPv6 connectivity via an HE 6in4 tunnel</h1>
<p>OpenWrt works with IPv6 out of the box. If your provider isn't supplying you with native v6 connectivity yet (which is unfortunately still very likely), you can use a <a href="http://www.tunnelbroker.net">free tunnel</a> provided by <a href="http://he.net">Hurricane Electric</a>. The registration process is pretty straightforward, and HE even provide configuration samples for many routers and operating systems. The one for OpenWrt didn't work for me (which was probably my fault, so you should give it a try!), but this one does:</p>
<p>First, the 6in4 tunneling module has to be installed:</p>
<div class="highlight"><pre>opkg install 6in4
</pre></div>
<p>Then, the tunnel interface can be configured in <code>/etc/config/network</code>.</p>
<p>For the local network, an IPv6 address has to be configured for the router. It can be any address in one of the "Routed IPv6 Prefixed" shown in the tunnel details on tunnelbroker.net, so if you were assigned the prefix <code>2001:470:1b:1234::/64</code>, you could choose <code>2001:470:1b:1234::1/64</code>.</p>
<div class="highlight"><pre>config interface 'lan'
option ifname 'eth0'
option type 'bridge'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
#
option ip6addr '2001:470:1b:1234:1/48'
</pre></div>
<p>The 6in4 tunnel interface has to be configured like this (assuming your "Client IPv6 address" is <code>2001:470:1a:1234::2</code>, your "Server IPv4 address" is <code>1.2.3.4</code>, and your "Tunnel ID" is <code>12345</code>):</p>
<div class="highlight"><pre>config interface 'henet'
option proto '6in4'
option peeraddr '1.2.3.4'
option ip6addr '2001:470:1a:1234::2/64'
option tunnelid '12345'
option username 'your.username'
option password 'yourpassword'
</pre></div>
<p>Username and password are <em>not</em> identical to your HE login credentials – they can be retrieved on the "Example Configurations" tab on the "Tunnel Details" page.</p>
<p>To configure the firewall, you can exceuted the command provided by HE (<code>uci set firewall.@zone[1].network='wan henet'</code>), or manually insert the line <code>option network 'wan henet'</code> to the <code>wan</code> block in the file <code>/etc/config/firewall</code>.</p>
<p>If you want to use one of IPv6's nicest features, stateless autoconfiguration, <code>radvd</code> has to be installed...</p>
<div class="highlight"><pre>opkg install radvd
</pre></div>
<p>...and configured like this (in <code>/etc/config/radvd</code>):</p>
<div class="highlight"><pre>config interface
option interface 'lan'
option AdvSendAdvert 1
option AdvManagedFlag 0
option AdvOtherConfigFlag 0
list client ''
option ignore 0
config prefix
option interface 'lan'
list prefix '2001:470:1b:1234::/64'
option AdvOnLink 1
option AdvAutonomous 1
option AdvRouterAddr 0
option ignore 0
</pre></div>
<h1>Dynamic DNS</h1>
<p>OpenWrt has built-in support for many dynamic DNS services, and the <a href="http://wiki.openwrt.org/doc/howto/ddns.client">documentation on their wiki</a> has all you need to configure it.</p>
<h1>Port forwarding</h1>
<p>Specific ports of devices behind the OpenWrt NAT can be made reachable by a block like this in <code>/etc/config/firewall</code>:</p>
<div class="highlight"><pre>config 'redirect'
option 'name' 'myhomeserver'
option 'src' 'wan'
option 'proto' 'tcp'
option 'src_dport' '22'
option 'dest_ip' '192.168.1.123'
option 'dest_port' '22'
option 'target' 'DNAT'
option 'dest' 'lan'
</pre></div>
<h1>Port mirroring with iptables</h1>
<p>My main motivation for upgrading to the release candidate was actually an iptables module called <code>tee</code>. It can be used to duplicate all or a subset of all packets going throuth the router and transmit them to some other host running a packet analyzer like Wireshark. This can be very useful to debug embedded wireless devices without having to resort to a Wifi card running as a packet sniffer, which was my previous approach.</p>
<p>In order for the following command to work, the <code>tee</code> module has to be installed on the router. For some reason, <code>ip6tables</code> is also required, or loading the module will fail:</p>
<div class="highlight"><pre>opkg install iptables-mod-tee ip6tables
</pre></div>
<p>To forward all traffic going through the router to a machine in the private network at 192.168.1.123, simply exceute this command in a shell:</p>
<div class="highlight"><pre>iptables -A POSTROUTING -t mangle -j TEE --gateway 192.168.1.123
</pre></div>
<p>The forwarded packets will still show the internal IP addresses, which makes finding a specific device much easier.</p>
<h1>To be continued</h1>
<p>That's my configuration for now – I hope to be able to update this blog regularly if I find some improvements to my setup.</p>Hello2013-01-13T12:00:00+01:00lxgrtag:blog.lxgr.net,2013-01-13:posts/2013/01/13/hello/<!--- Summary: Welcome to my blog -->
<p>This is going to be my new personal blog. Topics will vary from programming and technical stuff to random thoughts about (possibly non-technical) things.</p>