You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Feature/9 improved acquisition process of japanese tokens in contentbasedrecommenderts (#11)
* feat: Implement English and Japanese tokenizers with filtering capabilities
- Added EnglishTokenizer and JapaneseTokenizer classes for text tokenization.
- Integrated HTML tag stripping and case normalization in EnglishTokenizer.
- Implemented morphological analysis using kuromoji in JapaneseTokenizer.
- Created EnglishTokenFilter and JapaneseTokenFilter for token filtering.
- Developed a ProcessingPipelineFactory to create tokenization and filtering pipelines.
- Enhanced ContentBasedRecommender with improved training and recommendation logic.
- Added comprehensive unit tests for tokenizers, filters, and recommender functionality.
- Implemented integration tests for end-to-end functionality of the recommender system.
* Update package.json
---------
Co-authored-by: kemsakurai <sakurai.kem@mail.com>
0 commit comments