REDMINE_DMSF Summary of installing plugins in Linux under Bitnami-redmine

Installation Plugin REDMINE_DMSF Summary

• Background
• Environment
  
  • Linux system version
  • Redmine version
  • DMSF plugin Version
• installation process
  
  • Installing Redmine
  • Installing the DMSF Plugin
• Using full-text search in Redmine
  
  • Principles behind DMSF
    
    • Basic knowledge of Xapian and Omega
    • Create/Query Index database manually
  • DMSF Configuration Modifications
  • Using full-Text search
  • Automatically build index databases
• Summarize
• Appendix A summary of issues encountered in installing plug-ins
  
  • Unable to access https://rubygems.org
    
    • Workaround 1-Turn over the wall
    • Workaround 2-Use rubygems mirroring
    • Workaround 3-Manually download the source code compilation/Installation
  • Install Nokogiri dependent Library always error

Background

Redmine's own search function, can only search the wiki, unable to search the contents of the attachment file. For the company's big data, knowledge base, is unbearable. To find a plugin dmsf to solve this problem, DMSF supports full-text search (fulltext search).

Environment Linux System version

 
 
# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04 LTS
Release:    12.04
Codename:   precise
#
# cat /proc/version
Linux version 3.2.0-23-generic-pae ([email protected]) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu4) ) #36-Ubuntu SMP Tue Apr 10 22:19:09 UTC 2012
#

Redmine version

Bitnami-redmine-3.2.1-0-linux-installer.run

DMSF plugin Version

redmine_dmsf-1.5.6, called the "document steward" plugin in Chinese.

Installation process Installation Redmine

Installing the Bitnami-redmine-3.2.1-0-linux-installer.run process is simple and skips over.

Installing the DMSF Plugin

Under the Bitnami-redmine how to install plug-ins, first read "How to install a plugin on Redmine.doc" to enrich the foundation.

To install the DMSF plugin, refer to the documentation in the plugin, depending on the documentation requirements, you must first install some dependent tools:

# sudo apt-get install xapian-omega libxapian-dev xpdf xpdf-utils antiword unzip catdoc libwpd-tools libwps-tools gzip unrtf catdvi djview djview3 uuid uuid-dev xz-utils

It is also important to note that the redmine_dmsf-1.5.6 folder under the Plugins directory is renamed to REDMINE_DMSF as follows:

 
# tree -L 2 /opt/redmine-3.2.1-0/apps/redmine/htdocs/plugins /opt/redmine-3.2.1-0/apps/redmine/htdocs/plugins
├── README
└── redmine_dmsf
    ├── app
    ├── assets
    ├── CHANGELOG.md ├── config
    ├── db
    ├── dmsf_user_guide.odt ├── extra
    ├── Gemfile
    ├── init.rb ├── lib
    ├── README.md └── test 8 directories, 6 files

During the installation process, encountered some problems, recorded, see Appendix A.

Using full-text search in Redmine

DMSF plug-in installation, restart Redmine, the full-text search feature cannot be used at this time (that is, the attachment content cannot be searched), you need to first create the file index (including the file name index and document content index). DMSF is used to create indexes using the Xapian and Omega Tools. First understand the principle of Chu dmsf.

DMSF behind the scenes principles Xapian and Omega basics

Before continuing, come to know Xapian and Omega, citing the contents of IBM technical documents: Read and index documents with Xapian and Omega.

Storing and providing access to documents and information is an increasingly serious problem for many companies. There are many solutions to this problem, including wikis and structured document storage, but full-text indexing is often the only way to get information from a large number of documents. Xapian is an open source tool that can read and index documents, including HTML, PDF, OpenOffice, Microsoft? Office? Format, and also have some programmable interfaces to add and extract information, including Java? Technology, so that you can deploy the websphered? Document index is supported within the environment. In this article, learn how to install and deploy a typical Xapian to index various information, and then take a look at a few examples of extracting information using different language bindings. This process will focus on how to use it within a typical corporate intranet environment. This article will also provide a quick overview of Omega, a custom tool for collaborating with the Xapian infrastructure.

Xapian and Omega are independent two components designed to work together to provide indexing and search capabilities. The Xapian component provides core database functionality (storage information) and search and retrieval systems to find word and word combinations (see resources).

The Omega component provides tools to convert and parse information in different formats into the original format required by Xapian so that it can be indexed. Omega uses a variety of tools, such as Pdftotext, and then submits the converted and filtered text-based information so that the index and structure of the document can be identified and stored in the Xapian database. Omega is part of the Xapian download.

Local Folder "Xapian and Omega Basics" collects some basic knowledge, first of all, the basic knowledge of solid to continue, ax.

Create/Query Index database manually

In order to check the correctness of the plug-in installation, and to deepen the understanding of Xapian and Omega, first manually set up and query the index database.

• Establishing an Index database

#omindex--dbdbname--urlindexurl/home/files

DBName is a custom database name (which shows a folder name), Indexurl is the URL of the custom index information,/home/files is the file path you want to index. After execution, a folder is generated under the current path dbname, where is Indexurl? I'll see it right away.

• Querying the database
  Retrieves a document containing "information" from the database dbname.

# quest --db = dbname information
Parsed Query: Xapian :: Query (Z information: (pos = 1))
MSet:
67 [100%]
url = indexurl / f00153.txt
sample = Friday, April 22, 2016 * * * * * *
type = text / plain
modtime = 1604220123
size = 56841 

DMSF Configuration Modifications

After the DMSF plug-in is installed, ensure that the following two locations are configured consistently (see below):

1. Plug-in configuration: Management--Plug-in configuration dmsf
2. Configuration file:redmine_dmsf\extra\xapian_indexer.rb

With the theoretical basis of the previous chapters, change up to be able to do it, no one can not do it (I am in the configuration of this place this toss for a long time). Configuration "Redmine management, plug-in, dmsf" as shown, I use the default configuration here, without modification, but instead modify theredmine_dmsf\extra\xapian_indexer.rbfile to keep the two consistent.

andredmine_dmsf\extra\xapian_indexer.rbThe default configuration only suitable for pure redmine, not suitable for bitnami-redmine, forxapian_indexer.rbfile changes, compared to my local saved "xapian_indexer.rb_" and "Xapian_indexer.rb_ before", Can be at a glance. Change the following three variable definitions:

• $redmine _root
• • Role: Installation path of Redmine
  • Modify: Modify the path to Bitnami-redmine, such as:$redmine_root = ‘/opt/redmine-3.2.1-0/apps/redmine/htdocs‘

• $files

  
  

  • Function: The file storage path uploaded through the DMSF module. is also the path to the file where the index is to be indexed.
  • Modification: Requirements Red mINe_RooT/ The path to files is consistent with the file storage directory in Figure 1. I set this to:$files = ‘files/dmsf‘
• $dbrootpath
• • Role: The storage path of the index database. When searching by keyword, it is searched in this database.
  • Modification: The path required to $dbrootpath is consistent with the "Index database" in Figure 1. I set this to:$dbrootpath = ‘/opt/redmine-3.2.1-0/apps/redmine/htdocs/files/dmsf_index‘

Using full-Text search

The DMSF plugin description has the following description:

# # Fulltext Search (optional)
If you want to use Fulltext search features, you must setup file content indexing.

It is necessary to index DMSF files with Omega before searching attempts to receive some output:

1. Change the configuration part of redmine_dmsf/extra/xapian_indexer.rb file according to your environment.
2. Runruby redmine_dmsf/extra/xapian_indexer.rb -f

How to modify XAPIAN_INDEXER.RB has been described in the previous section, after the configuration, start the/opt/redmine-3.2.1-0/use_redmine console (you can not use the Linux system comes with the console, will not find the Ruby executable file), Execute the following command:

 
 
# /opt/redmine-3.2.1-0/use_redmine 
# cd /opt/redmine-3.2.1-0/apps/redmine/htdocs/plugins/
# ruby redmine_dmsf/extra/xapian_indexer.rb -v

Here is an additional description of Ruby's additional parameters:

# ruby xapian_indexer.rb -v # -v show the process, index the files and code warehouse of all projects
# ruby xapian_indexer.rb -v --f # -f only index files
# xapian_indexer.rb -v --r # -r only index the code warehouse
# xapian_indexer.rb -v -r -p proj_Id1, proj_Id2 # -p index specified items 

You can then use the full-text search feature in Redmine to search for the contents of the attachment file, as shown in.

Automatically build index databases

The previous index database is executed by manual entry commands. It is unrealistic to have someone upload a file at any time and manually enter a command to build an indexed database Redmine. So in DMSF's Readme document, the following description is the creation of cron Task timed Building Index Database (this block I have not studied, but this is not a good way ah, have not submitted the attachment document, you can sense, and automatically build the index database? Later study).

# # Fulltext Search (optional)
If you want to use Fulltext search features, you must setup file content indexing.

It is necessary to index DMSF files with Omega before searching attempts to receive some output:

1. Change the configuration part of redmine_dmsf/extra/xapian_indexer.rb file according to your environment.
2. Runruby redmine_dmsf/extra/xapian_indexer.rb -f

This command must is run on regular basis (e.g. from cron)

Example of cron Job (once per hour at 8th minute):

8 * * * * Root/usr/bin/ruby redmine_dmsf/extra/xapian_indexer.rb-f

See REDMINE_DMSF/EXTRA/XAPIAN_INDEXER.RB for help.

Summarize

The DMSF plugin brings full-text search functionality to Redmine, but in order to compare the accuracy and completeness of the search. I'm compared to confluence. (Confluence is a commercial Knowledge base management system), after some experience, found that redmine_dmsf-1.5.6 has the following deficiencies:

1. The same keyword, the same PDF document, can be searched on confluence, but not in the Redmine search. See DMSF Search integrity is not enough, do not know the future DMSF will solve the problem.
2. DMSF cannot do this, auto-sensing attachment uploads to automatically trigger the build file index. Can only do in the background to open a cron task, timed to trigger the construction of file index, real-time poor. (For this, it is possible that I did not study in depth, if I was wrong, please correct me!) ）

Appendix A summary of issues encountered by the installation plug-in cannot be accessed https://rubygems.org

During the execution of the bundle Install–without development test PostgreSQL sqlite–no-deployment, the dependencies of the plugins in the plugins directory are checked and the dependent libraries are automatically downloaded and installed.

This installation, there is no access to the https://rubygems.org, causing the dependent library cannot download the installation. The error description is as follows:

Gem::RemoteFether::FetchError: Errno::ECONNABORTEDbyin your host machine.- SSL_connect (https://rubygems.org/gems/simple_enum-2.3.0.gem)

How to solve this problem, there are several workarounds:

Workaround 1-Turn over the wall

RubyGems has been very difficult to access at home, in the local you may be able to turn over the wall, when you want to publish on-line, the server can not be allowed to turn over the wall Oh, you are very difficult to engage! All of this is locally appropriate and inappropriate on the server.

Workaround 2-Use rubygems mirroring

Because of the existence of the wall, the "RubyGems Mirror" was spawned. Official website https://rubygems.org can not access, that is used mirror, such as https://gems.ruby-china.org/(mirror over time, it is possible to change, their own Baidu "RubyGems Mirror" search).
You can use the Bundler Gem source code to mirror the command settings as follows.

# bundle config mirror.https: //rubygems.org https://gems.ruby-china.org
# bundle config # Note: After setting, you can use this command to check the effect of the setting 

Workaround 3-Manually download the source code compilation/Installation

Go to Https://rubygems.org/gems to findsimple_enumthe specified version (heresimple_enum-2.3.0.zip), download the source code to compile:gem build simple_enum.gemspec, (to launchuse_redmine.batinto a specific shell, You can't use the CMD and shell that comes with Windows or Linux, and the commands are as follows:

 
# gem build simple_enum.gemspec Successfully built RubyGem
  Name: simple_enum
  Version: 2.3.0 File: simple_enum-2.3.0.gem #

Install after successful compilation:

 
 
# gem install --local simple_enum-2.3.0.gem
Successfully installed simple_enum-2.3.0
Parsing documentation for simple_enum-2.3.0
Installing ri documentation for simple_enum-2.3.0
Done installing documentation for simple_enum after 0 seconds
1 gem installed

Install Nokogiri dependent Library always error

The first is a hint that the g++ compiler is not installed (because g++ is required for compiling Nokogiri). This problem is solved by installing g++.

After installing g++, the compilation process prompts you to find a header file and a link library. Resolve by adding the compile option –use-system-libraries, as shown below, note:

# bundle config # First use this command to view the original compilation options
# bundle config build.nokogiri "--use-system-libraries --with-opt-dir = / opt / redmine-3.2.1-0 / common" # Add --use-system-libraries Comes with it) # bundle config # Note: After setting, you can use this command to check the effect of the setting 

REDMINE_DMSF Summary of installing plugins in Linux under Bitnami-redmine