Compile a simple FUSE File System tutorial in Python,

Compile a simple FUSE File System tutorial in Python,

If you are a long-term reader of mine, you should know that I am looking for a perfect backup program, and finally I wrote my own encryption layer based on bup.

When writing encbup, I am not satisfied with the practice that only one file is restored and the entire huge file must be downloaded, however, we still want to use EncFS and rdiff-backup together to implement remote mount, encryption, deduplication, and version backup.

After trying obnam again, I noticed that it had a mount command. After in-depth research, I found fuse-python and fusepy, and I feel it is quite easy to write a FUSE File System in Python.

Smart readers may have realized what I want to do next: I decided to write an encrypted file system layer in Python! It is very similar to EncFS, but there are also some important differences:

• It runs in reverse mode by default. It receives normal files and exposes an encrypted directory. Any backup program will discover (and back up) these encrypted directories without any other storage.
• It can also accept configuration files consisting of a directory list and expose these directories at the Mount point. In this way, all backup scripts need to back up the mount point, and different directories will be backed up immediately.
• It focuses more on backup than encrypted storage. It should be interesting to write.

Example of a FUSE File System

The first step in writing this script is to write a purely transfer-type file system. It only accepts a directory and exposes it at the Mount point to ensure that any modifications made to the mount point will be mirrored to the source data.

Fusepy requires you to write a class that defines various operating system-level methods. You can choose to define the methods that your file system wants to support. Others can not be defined for the moment, but I need to define all the methods, because my file system is a transfer-type file system, it should behave as consistent as possible with the original file system.

Writing this code is very simple and interesting, because most of the methods are simple encapsulation of the OS module (indeed, you can assign values to them directly, such as open = OS. open and so on, but my module needs some path extensions ). Unfortunately, fuse-python has a bug (as far as I know) that when opening and reading a file, it cannot return the file handle to the file system. Therefore, my script does not know the file handle corresponding to the read/write operation of an application, resulting in a failure. You only need to make a few changes to fusepy, and it can run well. It only has one file, so you can put it directly into your project.
Code

Here, I am happy to provide this code. You can refer to it when you plan to implement the file system on your own. This Code provides a good starting point. You can copy this class directly to your project and rewrite some of the methods as needed.

The following is the real code:

#!/usr/bin/env python from __future__ import with_statement import osimport sysimport errno from fuse import FUSE, FuseOSError, Operations class Passthrough(Operations):  def __init__(self, root):    self.root = root   # Helpers  # =======   def _full_path(self, partial):    if partial.startswith("/"):      partial = partial[1:]    path = os.path.join(self.root, partial)    return path   # Filesystem methods  # ==================   def access(self, path, mode):    full_path = self._full_path(path)    if not os.access(full_path, mode):      raise FuseOSError(errno.EACCES)   def chmod(self, path, mode):    full_path = self._full_path(path)    return os.chmod(full_path, mode)   def chown(self, path, uid, gid):    full_path = self._full_path(path)    return os.chown(full_path, uid, gid)   def getattr(self, path, fh=None):    full_path = self._full_path(path)    st = os.lstat(full_path)    return dict((key, getattr(st, key)) for key in ('st_atime', 'st_ctime',           'st_gid', 'st_mode', 'st_mtime', 'st_nlink', 'st_size', 'st_uid'))   def readdir(self, path, fh):    full_path = self._full_path(path)     dirents = ['.', '..']    if os.path.isdir(full_path):      dirents.extend(os.listdir(full_path))    for r in dirents:      yield r   def readlink(self, path):    pathname = os.readlink(self._full_path(path))    if pathname.startswith("/"):      # Path name is absolute, sanitize it.      return os.path.relpath(pathname, self.root)    else:      return pathname   def mknod(self, path, mode, dev):    return os.mknod(self._full_path(path), mode, dev)   def rmdir(self, path):    full_path = self._full_path(path)    return os.rmdir(full_path)   def mkdir(self, path, mode):    return os.mkdir(self._full_path(path), mode)   def statfs(self, path):    full_path = self._full_path(path)    stv = os.statvfs(full_path)    return dict((key, getattr(stv, key)) for key in ('f_bavail', 'f_bfree',      'f_blocks', 'f_bsize', 'f_favail', 'f_ffree', 'f_files', 'f_flag',      'f_frsize', 'f_namemax'))   def unlink(self, path):    return os.unlink(self._full_path(path))   def symlink(self, target, name):    return os.symlink(self._full_path(target), self._full_path(name))   def rename(self, old, new):    return os.rename(self._full_path(old), self._full_path(new))   def link(self, target, name):    return os.link(self._full_path(target), self._full_path(name))   def utimens(self, path, times=None):    return os.utime(self._full_path(path), times)   # File methods  # ============   def open(self, path, flags):    full_path = self._full_path(path)    return os.open(full_path, flags)   def create(self, path, mode, fi=None):    full_path = self._full_path(path)    return os.open(full_path, os.O_WRONLY | os.O_CREAT, mode)   def read(self, path, length, offset, fh):    os.lseek(fh, offset, os.SEEK_SET)    return os.read(fh, length)   def write(self, path, buf, offset, fh):    os.lseek(fh, offset, os.SEEK_SET)    return os.write(fh, buf)   def truncate(self, path, length, fh=None):    full_path = self._full_path(path)    with open(full_path, 'r+') as f:      f.truncate(length)   def flush(self, path, fh):    return os.fsync(fh)   def release(self, path, fh):    return os.close(fh)   def fsync(self, path, fdatasync, fh):    return self.flush(path, fh) def main(mountpoint, root):  FUSE(Passthrough(root), mountpoint, foreground=True) if __name__ == '__main__':  main(sys.argv[2], sys.argv[1])

If you want to run it, you only need to install fusepy, put this code into a file (such as myfuse. py), and then run python myfuse. py/your directory/mount point directory. You will find that all the files in the "/your directory" path are running to the "/mount point directory" and can operate them like they are using a native file system.
Conclusion

In general, I don't think writing a file system is that simple. The next step is to add the encryption/Decryption function in the script and Some helper methods. My goal is to make it a complete alternative to EncFS in addition to better scalability (because it was written in Python) and some additional features for backup files.

If you want to follow up the development process of this script, subscribe to my email list below or follow me on Twitter. As always, you are welcome to give feedback (it is good to comment below ).