How to open a large text file in Linux

In the "Big Data" era, we often encounter large text files (larger or larger. Suppose we need to manually search for and edit these large files, or to solve some specific problems, we need to manually analyze multiple log files on GB. Traditional text editing software is not very effective in processing such large files. When we try to open a large file, it is often frustrated by insufficient memory.

If you are a savvy System Administrator, you may use a combination of cat, tail, grep, sed, awk, and other commands to open and edit a text file. In this tutorial, I will talk about how to open (and edit) a large text file in Linux in a more friendly way.

Vim LargeFile plug-in

The Vim text editor has a large number of plug-ins (or scripts) that can expand VIM functions. One of the Vim plug-ins is the LargeFile plug-in.

The LargeFile plug-in allows you to quickly load and edit large files. It is implemented by disabling VIM functions such as events, rollback, and syntax highlighting.

To install the LargeFile plug-in on VIM, first confirm whether VIM has been installed.

In Debian, Ubuntu, or Linux Mint:

$ sudo apt-get install vim 

In the Fedora, CentOS, or RHEL system:

$ sudo yum install vim-enhanced 

You can download the LargFile plug-in from Vim website. The latest version number is 5. The downloaded file will be saved in Vimball format (ended with. vba ).

To install the plug-in your home directory, use VIM to open the. vba file, as shown below.

$ gunzip LargeFile.vba.gz$ vim LargeFile.vba 

Enter ": so %" in the VIM form and press enter to install the plug-in your home directory.

Enter ": q" to exit VIM.

This plug-in will be installed in ~ /. Vim/plugin/LargeFile. vim. Now you can use VIM as usual.

When a "large" file is loaded in VIM, this plug-in is used to disable events, rollback, syntax highlighting, and other functions. By default, files larger than 100 MB are considered as "large files" by the plug-in ". To change the default settings, You can edit the settings ~ /. Vimrc file (create one if it does not exist ).

To minimize the size of a large file to 10 MB ~ /. Add in vimrc

let g:LargeFile=10

Although LargeFile can speed up file loading, VIM itself does not support editing a large file, because it will load the entire file into the memory at once. For example, if you use VIM to load a 1 GB file, it will occupy a lot of memory and swap space, as shown in the top output.

Therefore, if your file is significantly larger than the physical memory of your Linux system, consider other options, as described below.

1 2 Next Page