With Golang, a file monitor, a sync gadget.

This article was first published on Rootrl's Blog

Origin

Our company development environment is very special, a local server, and then assign multiple SSH accounts to the developer. Usually upload code can only ftp/sftp connection upload (previously used samba sharing, but was turned off). ）。 So we usually use SFTP on the phpstorm to remotely open the items on the server, and then set up automatic uploads. It's not a problem to work in general. But there's a pit: Phpstorm can't capture changes like git checkout these change files. So there's no way to keep the local code consistent with the server. So it was born to want to write this synchronization mechanism of the idea, this scene golang very suitable. So he started to masturbate.

Project Address: Https://github.com/rootrl/Mancy

Realize

The general idea is to monitor the change of a folder (local code base) and upload it to the server via SFTP if there is a change.

Monitoring file changes are used by the Golang fsnotify package, which provides the following types of monitoring changes:

const (    FSN_CREATE = 1    FSN_MODIFY = 2    FSN_DELETE = 4    FSN_RENAME = 8    FSN_ALL = FSN_MODIFY | FSN_DELETE | FSN_RENAME | FSN_CREATE)

But Fsnotify has a pit that can only monitor changes in a layer of folders, and multi-tiered folders need to traverse the Mount event themselves. Subsequent to the new folder, rename this also to manually add the event.

Each of these events corresponds to a processing channel, and my idea is to decouple the file processor from the event, since it is not necessarily sftp to handle uploads, or rsync, or other processing methods. So the subsequent processor only listens to the corresponding event channel, the processor here I use the Golang Select to implement a timeout mechanism, there are events on the processing, no event has a few seconds of waiting time.

Fsnotify this piece of code see: Https://github.com/rootrl/Man ...
SFTP hanlder See: Https://github.com/rootrl/Man ...

SFTP with the Github.com/pkg/sftp this library, used to be quite handy, but all are writing the underlying API, so I individually encapsulated a sftp_util:https://github.com/rootrl/man ... There are some common upload files/folders, delete files/folders and other operations

The above basically can achieve the main function, and then I also defined a configuration file structure, through the corresponding JSON characters can be automatically mapped to this structure, for subsequent use. This is also the convenience of the Golang JSON package.

Summarize

Write this project is mainly used to practiced hand golang, at the beginning of the stage, the code may be written a bit rubbish. For example, Sftpclent client these are currently implemented with global variables, but are not elegant enough to be used. Follow up slowly to improve. (may never be.) ）

In short, Golang is very good!