To install the gem remotely from RubyForge:

sudo gem install rmmseg-cpp

Or you can download the gem file manually from RubyForge and install it locally:

sudo gem install —local rmmseg-cpp-x.y.z.gem

To build the gem manually from the latest source code. You’ll need to have git and rake installed.

The source code of rmmseg-cpp is hosted at GitHub. You can get the source code by git clone:

git clone git://github.com/pluskid/rmmseg-cpp.git

then you can use Rake to build and install the gem:

cd rmmseg-cpp rake gem:install

To use rmmseg-cpp in Ruby program, you’ll first load it with RubyGems:

require 'rubygems'
require 'rmmseg'

Then you may customize the dictionaries used by rmmseg-cpp (see the rdoc on how to add your own dictionaries) and load all dictionaries:

RMMSeg::Dictionary.load_dictionaries

Now rmmseg-cpp will be ready to do segmenting. If you want to load your own customized dictionaries, please customize RMMSeg::Dictionary.dictionaries before calling load_dictionaries. e.g.

RMMSeg::Dictionary.dictionaries = [[:chars, "my_chars.dic"],
                                   [:words, "my_words.dic"],
                                   [:words, "my_words2.dic"]]

The basic format for char-dictionary and word-dictionary are similar. For each line, there is a number, then a space, then the string. Note there SHOULD be a newline at the end of the dictionary file. And the number in char-dictionary and word-dictionary has different meaning.

In char-dictionary, the number means the frequency of the character. In word-dictionary, the number mean the number of characters in the word. Note that this is NOT the number of bytes in the word.

To use rmmseg-cpp with Ferret, you’ll need to require the Ferret support of rmmseg-cpp (Of course you’ll also have to got Ferret installed. If you have problems running the belowing example, please try to update to the latest version of both Ferret and rmmseg-cpp first):

require 'rmmseg/ferret'

rmmseg-cpp comes with a ready to use Ferret analyzer:

analyzer = RMMSeg::Ferret::Analyzer.new { |tokenizer|
  Ferret::Analysis::LowerCaseFilter.new(tokenizer)
}
index = Ferret::Index::Index.new(:analyzer => analyzer)

A complete example can be found in misc/ferret_example.rb. The result of running that example is shown in Figure 1. Ferret Example Screenshot.

rmmseg-cpp can also be used in normal Ruby programs. Just create an Algorithm object and call next_token until a nil is returned:

algor = RMMSeg::Algorithm.new(text)
loop do
  tok = algor.next_token
  break if tok.nil?
  puts "#{tok.text} [#{tok.start}..#{tok.end}]"
end