A tutorial on writing a simple fuse file system in Python

If you are my long-term reader, then you should know I am looking for a perfect backup program, and finally I wrote a bup based on my own encryption layer.

While writing Encbup, I'm not satisfied with the need to download the entire archive file just to recover a file, but I still want to be able to use ENCFS and rdiff-backup for remote mount, encryption, de-load, and versioned backup functions.

After trying Obnam again (wordy: It's still surprisingly slow), I noticed that it had a mount command. After a thorough study, I found Fuse-python and fusepy, feeling that writing a fuse file system in Python should be quite simple.

The smart reader may have realized what I was going to do next: I decided to write an encrypted file system layer in Python! It is very similar to ENCFS, but there are some important differences:

• It runs in reverse mode by default, receives normal files, and exposes an encrypted directory. Any backup program discovers (and backs up) these encrypted directories and does not require any additional storage.
• It can also accept a configuration file that consists of a directory list and exposes those directories at the mount point. In this case, all backup scripts will need to have the mount point back up, and the various directories will be backed up immediately.
• It will be more focused on backups than on encrypted storage. It should be interesting to write.

Example of a fuse file system

The first step in writing this script is to write a purely transitive file system. It simply accepts a directory and exposes it to the mount point, ensuring that any modifications to the mount points are mirrored in the source data.

Fusepy requires you to write a class that defines the various operating system-level methods. You can choose to define the methods that your file system wants to support, others can be temporarily undefined, but I need to define all the methods because my filesystem is a transitive filesystem that should behave as well as possible with the original filesystem.

Writing this code can be very simple and interesting, because most of the methods are just some simple encapsulation of the OS modules (you can assign them directly, such as Open=os.open and so on, but my module needs some path extensions). Unfortunately, Fuse-python has a bug (as far as I know) that when you open and read a file, it cannot pass the file handle back to the file system. Thus, my script does not know which file handle is corresponding to the read and write operation of an application, resulting in a failure. You just need to make minimal changes to the fusepy and it will work well. It has only one file, so you can put it directly into your project.
Code

Here, I am happy to give this code, when you intend to implement the file system you can refer to. This code provides a good starting point for you to copy the class directly into your project and rewrite some of the methods as needed.

The following is the real code:

#!/usr/bin/env python from __future__ import with_statement import osimport sysimport errno from fuse import fuse, fuseose Rror, Operations class Passthrough (Operations): Def __init__ (self, root): Self.root = root # Helpers # ======= de F _full_path (self, partial): If Partial.startswith ("/"): Partial = partial[1:] path = Os.path.join (Self.root, p artial) return Path # Filesystem methods # ================== def access (self, path, mode): Full_path = Self._f Ull_path (path) if not os.access (Full_path, mode): Raise Fuseoserror (errno. eacces) def chmod (self, Path, mode): Full_path = Self._full_path (path) return Os.chmod (Full_path, mode) def Chow N (self, path, UID, gid): Full_path = Self._full_path (path) return Os.chown (Full_path, UID, GID) def getattr (self, Path, fh=none): Full_path = Self._full_path (path) st = Os.lstat (Full_path) return Dict ((Key, GetAttr (St, key)) fo R key in (' St_atime ', ' st_ctime ', ' st_gid ', ' St_mode ', ' St_mtime ', ' st_nlink ', ' st_size ', ' St_uid ') def readdir (self, Path, FH): Full_path = Self._full_path (path) dir    Ents = ['. ', ' ... '] If Os.path.isdir (Full_path): Dirents.extend (Os.listdir (Full_path)) for R in Dirents:yield R def Readlink (s  Elf, path): pathname = Os.readlink (Self._full_path (path)) if Pathname.startswith ("/"): # path name is absolute,      Sanitize it. Return Os.path.relpath (pathname, Self.root) Else:return pathname def mknod (self, path, mode, dev): Return OS . Mknod (Self._full_path (path), mode, dev) def rmdir (self, path): Full_path = Self._full_path (path) return Os.rmdir ( Full_path) def mkdir (self, Path, mode): Return Os.mkdir (Self._full_path (path), mode) def statfs (self, path): ful L_path = Self._full_path (path) STV = OS.STATVFS (Full_path) return Dict ((Key, GetAttr (STV, key)) for key in (' F_bavai L ', ' f_bfree ', ' f_blocks ', ' f_bsize ', ' f_favail ', ' f_ffree ', ' f_files ', ' f_flag ', ' f_frsize ', 'F_namemax ') def unlink (self, path): Return Os.unlink (Self._full_path (path)) def symlink (self, Target, name): RE Turn Os.symlink (Self._full_path (target), Self._full_path (name)) def rename (self, old, new): Return Os.rename (Self._fu Ll_path (old), Self._full_path (new)) def link (self, target, name): Return Os.link (Self._full_path (target), Self._full_ Path (name)) def utimens (self, Path, Times=none): Return Os.utime (Self._full_path (path), times) # File Methods # = = ========== def open (self, Path, flags): Full_path = Self._full_path (path) return Os.open (Full_path, flags) def c Reate (self, path, mode, fi=none): Full_path = Self._full_path (path) return Os.open (Full_path, OS. o_wronly | Os. O_creat, mode) def read (self, path, length, offset, FH): Os.lseek (FH, offset, OS. Seek_set) return Os.read (FH, length) def write (self, path, buf, offset, FH): Os.lseek (FH, offset, OS. Seek_set) return Os.write (FH, BUF) def truncate (self, path, length, Fh=noNE): full_path = Self._full_path (path) with open (Full_path, ' r+ ') as-f:f.truncate (length) def flush (self, PA Th, FH): Return Os.fsync (FH) def release (self, Path, FH): Return Os.close (FH) def fsync (self, path, Fdatasync, F h): Return Self.flush (Path, FH) def main (Mountpoint, root): FUSE (Passthrough (Root), Mountpoint, foreground=true) if __ name__ = = ' __main__ ': Main (sys.argv[2], sys.argv[1])

If you want to run it, just install fusepy, put the code in a file (like myfuse.py) and run the Python myfuse.py/your directory/mount point directory. You will find that all files under the "/Your Directory" path go to the "/Mount point directory" and can also manipulate them as if they were native file systems.
Conclusion

Overall, I don't think it's that simple to write a file system. The next thing to do is to add the encryption/decryption functionality to the script, as well as some helper classes. My goal is to make it more extensible (because it's written in Python) and include some extra features for backup files, which can be a complete replacement for a ENCFS.

If you want to follow up on the development of this script, please subscribe to my mailing list below, or follow me on Twitter. As always the welcome feedback (in the comments below is very good).

