tree 78f1323dd7a614cb0011d08717ef64ca116f9e49
parent 25b997d798dbe08d2b2dbd9414ed3747e1918889
author Noriyuki Takahashi <noriyukit@google.com> 1422695292 +0000
committer Yohei Yukawa <yukawa@google.com> 1422695292 +0000

Refactor SystemDictionary::PrefixSearch using LoudsTrie::Node APIs

This CL re-implements prefix search without LoudsTrie::Callback, which simplifies code because the nested callback of DictionaryInterface::Callback and LoudsTrie::Callback was removed.

Also, some performance improvements are implemented.
- Instead of decoding the prefix using Codec::DecodeKey, only its length is decoded by Codec::GetDecodeKeyLength and the decoded prefix is obtained from the original user input.  This is faster because we can avoid UCS4 to UTF8 conversion and string construction.
- Lazy decoding of LoudsTrie's key ID.  In the previous implementation, key IDs were always decoded due to the restriction of LoudsTrie::Callback.  However, key ID is first used when starting decoding token array elements.  Thus, when performing culling based on key strings, we can skip decoding of key ID.
- Optimization for the case where key expansion is not performed (e.g., predictor calls LookupPrefix without expansion).  In the previous implementation, this case was implemented by passing the default KeyExpansionTable.  However, since recursive traversal over LoudsTrie is far complicated than the normal traversal, it's worth optimizing the code for non-expanding cases.

Roughly 10% time reduction is observed in converter_benchmark with several corpora.  Note that this is just a code refactoring, no user-visible behavior change is intended.

BUG=none
TEST=unittest,benchmark

git-svn-id: https://mozc.googlecode.com/svn/trunk@525 a6090854-d499-a067-5803-1114d4e51264