Towards Application Driven Storage. Optimizing RocksDB for Open-Channel SSDs

Posted on by

Open-channel SSDs enable the host to undertake responsibilities that in typical SSDs are assumed by the Flash Translation Layer (FTL) embedded in the device. From the application perspective, this means that storage policies that have traditionally been based on best efforts can now be tailored for each specific application. In other words, applications can implement their own FTLs and govern data placement and garbage collection.

In this talk at LinuxCon Europe, I explain how to integrate an application FTL with LightNVM – the kernel subsystem providing support for Open-Channel SSDs. More specifically, I report on the design and implementation of a LightNVM backend for RocksDB we refer to as DFlash (Direct Flash). The focus is on both the LightNVM interfaces to which other applications could hook to, and the modifications to RocksDB so that it can communicate with LightNVM. I also provide some experimental results on a real hardware CNEX Labs Open-Channel SSD.

If you have any questions or feedback feel free to write to me at javier<at>javigon.com

Presentation PDF.

Enjoy! 🙂

A simple way to keep sensitive data safe in Linux

Posted on by

Encrypting Linux’s home directory is usually the first thing people worried about their personal data do. However, this approach comes at the cost of: (i) slower booting, (ii) problems when rebooting remotely and ssh-ing the machine straightway (more about this in the post), and (iii) the necessity to place all sensitive information in the home directory. In this post I will show another way to keep sensitive information encrypted, which – in my view -, is simpler and more secure.

Continue reading →

Enabling VPN while preserving external access in your home NAS

Posted on by

Enabling VPN in a home NAS is a good way – some might say paramount requirement – to preserve your privacy when using certain applications. While the highest flexibility comes from running the NAS directly in a Linux machine, out-of-the-box solutions can win in convenience thanks to built-in applications and mobile App support. One of such solutions is Synology. In this post I will describe how to setup OpenVPN in a Synology NAS while still preserving access from the outside.

Continue reading →

Using reverse ssh tunnels for efficient remote working

Posted on by

Having in place a setup that allows remote access to your development machine – which most probably is located at your work place -, has many advantages. Not only does it prevent duplicating equipment (e.g., having a decent set-top box at home), but it enables the possibility to get work done when travelling. However, some times it is desirable that such development machine is only available from a local network. It is in these cases that reverse ssh tunnels come handy.

Continue reading →

Automating fio tests with Python

Posted on by

Lately, I am spending quite some time verifying Open-Cannel SSD hardware prototypes in the context of LightNVM. A great deal of these tests is about benchmarking performance (iops) and latency, for which I use fio. Fio is a great tool to test specific I/O patterns. However, when wanting to benchmark a range of I/O configurations in different setups, it is very convenient to automatize the process. In this post, I present a simple python script that allows to automatize fio tests as a function of block size, number of jobs, I/O depth, etc. and stores the test results in csv format. Also, in order to account for variances, the script supports replicating each fio test a configurable number of times, being the average of these the one being stored.

Continue reading →

Setup for transparent SSH with iTerm and AppleScript

Posted on by

I use my MacBook as my main machine from which I access daily a number of servers that I use for development, experimentation, and maintenance. Since I do all my job from the terminal, I end up using SSH extensively. One major issue I have encountered is that SSHing different servers entails (i) having a mental map of which terminals correspond to which machines, and (ii) the overhead of introducing a SSH line immediately after opening a new terminal window. Although this can be mitigated by merely using zsh’s substring history search, aliases, etc., I found it easier to offload the responsibility to my muscle memory. In this post, I will describe my setup which, after over a month of being in production, has proved to make my life easier.

Continue reading →

Loading iTunes library from command line

Posted on by

Having separated iTunes libraries for different setups (e.g., media center, local music, remote playlists) is a powerful way to organize your media library and share resources among different devices (e.g., iPhone synchronization), but it can be painful. While it is possible to load a different iTunes library by opening iTunes and pressing “Alt” at the same time, this is not optimal. Ideally, we would write a script to pass iTunes the library it should load at launch time (e.g., open /Applications/iTunes.app –load-library=$LIBRARY_PATH). However, to the best of my knowledge, the iTunes API does not expose this – which I honestly do not understand. Googling for a workaround I could not find a good/complete solution, so I made my own. Here you have the setup and the scripts that implement it.

Continue reading →

Maintaining an updated version of vim

Posted on by

I use vim for development. More concretely, I use YADR, a fork of dotfiles for my zsh, git, vim, etc. setup. One of the vim plugins that I use the most is neocomplete, which provides simple but powerful autocompletion. One of the last commits in neocomplete introduces a problem that impedes correctly wrapping lines – normally set to 80 characters (issue here). It turns out that it was vim, not neocomplete that triggered this unwanted behaviour. This vim patch, fixes the issue, but the binary cannot be obtained for neither MacOS nor Ubuntu – which are my main OSs – via standard repositories. Thus, I decided to find a way to keep my vim updated so that I can easily patch future problems.

Continue reading →

Certainty Boot

Posted on by

A month ago we presented our work on a new technique for validating an execution environment at boot time at ISC’14; we refer to it as certainty boot. In the first stages of this work, we found inspiration in Cory Doctorow’s talk “The Coming War on General Purpose Computing”. Here, Doctorow actually proposes a high-level description of what we have implemented. At the moment we have a prototype based on ARM TrustZone as a Trusted Execution Environment and a Secure Element as a tamper-resistant unit. Now we are working on a demo using a Zynq ZC702 development board.

Here you can find the article and the presentation.

Continue reading →

Configuring kmemleak on the Zynq

Posted on by

The other day I came across a memory leak in a char device driver I am implementing for the Linux Kernel. While in user space one can make use of Valgrind (see post), kernel space is not that simple to monitor. Instead, one can use kmemleak, a memory leak detector embedded in the kernel which, among other architectures, supports ARM. In this post I will cover one minor configuration obstacle that one must overcome to get kmemleak to work on the Zynq ZC702.

Continue reading →