Skip to content

Commit 2ad05a2

Browse files
boyan01boyan01
authored
add string_tokenizer package (#228)
* add string tokenizer * update documents * update * update readme
1 parent e659492 commit 2ad05a2

File tree

83 files changed

+9837
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

83 files changed

+9837
-0
lines changed

‎README.md

+2
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ Any questions join the [Mixin Flutter group][mixin_group_url]
2020
| [fts5_simple](./packages/fts5_simple) | [![pub package][fts5_simple_version]][fts5_simple_pub] | [![likes][fts5_simple_likes]][fts5_simple_score] | Windows, Linux, macOS |
2121
| [desktop_multi_window](./packages/desktop_multi_window) | [![pub package][desktop_multi_window_version]][desktop_multi_window_pub] | [![likes][desktop_multi_window_likes]][desktop_multi_window_score] | Windows, Linux, macOS |
2222
| [mixin_logger](./packages/mixin_logger) | [![pub package][mixin_logger_version]][mixin_logger_pub] | [![likes][mixin_logger_likes]][mixin_logger_score] | All Platforms |
23+
| [ui_device](./packages/ui_device) | NONE | NONE | macOS, iOS |
24+
| [string_tokenizer](./packages/string_tokenizer) | NONE | NONE | macOS, iOS |
2325

2426
[desktop_drop_version]: https://img.shields.io/pub/v/desktop_drop.svg
2527

‎packages/string_tokenizer/.gitignore

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Miscellaneous
2+
*.class
3+
*.log
4+
*.pyc
5+
*.swp
6+
.DS_Store
7+
.atom/
8+
.buildlog/
9+
.history
10+
.svn/
11+
migrate_working_dir/
12+
13+
# IntelliJ related
14+
*.iml
15+
*.ipr
16+
*.iws
17+
.idea/
18+
19+
# The .vscode folder contains launch configuration and tasks you configure in
20+
# VS Code which you may wish to be included in version control, so this line
21+
# is commented out by default.
22+
#.vscode/
23+
24+
# Flutter/Dart/Pub related
25+
# Libraries should not include pubspec.lock, per https://dart.dev/guides/libraries/private-files#pubspeclock.
26+
/pubspec.lock
27+
**/doc/api/
28+
.dart_tool/
29+
.packages
30+
build/

‎packages/string_tokenizer/.metadata

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# This file tracks properties of this Flutter project.
2+
# Used by Flutter tool to assess capabilities and perform upgrades etc.
3+
#
4+
# This file should be version controlled.
5+
6+
version:
7+
revision: 4f9d92fbbdf072a70a70d2179a9f87392b94104c
8+
channel: stable
9+
10+
project_type: plugin_ffi
11+
12+
# Tracks metadata for the flutter migrate command
13+
migration:
14+
platforms:
15+
- platform: root
16+
create_revision: 4f9d92fbbdf072a70a70d2179a9f87392b94104c
17+
base_revision: 4f9d92fbbdf072a70a70d2179a9f87392b94104c
18+
- platform: ios
19+
create_revision: 4f9d92fbbdf072a70a70d2179a9f87392b94104c
20+
base_revision: 4f9d92fbbdf072a70a70d2179a9f87392b94104c
21+
22+
# User provided section
23+
24+
# List of Local paths (relative to this file) that should be
25+
# ignored by the migrate tool.
26+
#
27+
# Files that are not part of the templates will be ignored by default.
28+
unmanaged_files:
29+
- 'lib/main.dart'
30+
- 'ios/Runner.xcodeproj/project.pbxproj'

‎packages/string_tokenizer/CHANGELOG.md

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
## 0.0.1
2+
3+
* initial release.

‎packages/string_tokenizer/LICENSE

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2023 Mixin Network
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎packages/string_tokenizer/README.md

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# string_tokenizer
2+
3+
A Flutter package that provides a utility method for breaking text into individual words.
4+
5+
## Usage
6+
7+
To use this package, add `string_tokenizer` as
8+
a [dependency in your pubspec.yaml file](https://flutter.dev/docs/development/packages-and-plugins/using-packages).
9+
10+
```yaml
11+
dependencies:
12+
string_tokenizer:
13+
git:
14+
url: https://github.com/MixinNetwork/flutter-plugins.git
15+
path: packages/string_tokenizer
16+
```
17+
18+
Then import the package:
19+
20+
```dart
21+
import 'package:string_tokenizer/string_tokenizer.dart';
22+
```
23+
24+
### Breaking Text Into Words
25+
26+
`string_tokenizer` provides a single method `tokenize` which takes a string and breaks it into individual words.
27+
28+
```dart
29+
30+
List<String> words = tokenize('This is a sentence.');
31+
```
32+
33+
The `words` list will contain the following strings:
34+
35+
```log
36+
['This', 'is', 'a', 'sentence']
37+
```

‎packages/string_tokenizer/analysis_options.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
include: package:flutter_lints/flutter.yaml
2+
3+
# Additional information about this file can be found at
4+
# https://dart.dev/guides/language/analysis-options

‎packages/string_tokenizer/example/.gitignore

+44
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Miscellaneous
2+
*.class
3+
*.log
4+
*.pyc
5+
*.swp
6+
.DS_Store
7+
.atom/
8+
.buildlog/
9+
.history
10+
.svn/
11+
migrate_working_dir/
12+
13+
# IntelliJ related
14+
*.iml
15+
*.ipr
16+
*.iws
17+
.idea/
18+
19+
# The .vscode folder contains launch configuration and tasks you configure in
20+
# VS Code which you may wish to be included in version control, so this line
21+
# is commented out by default.
22+
#.vscode/
23+
24+
# Flutter/Dart/Pub related
25+
**/doc/api/
26+
**/ios/Flutter/.last_build_id
27+
.dart_tool/
28+
.flutter-plugins
29+
.flutter-plugins-dependencies
30+
.packages
31+
.pub-cache/
32+
.pub/
33+
/build/
34+
35+
# Symbolication related
36+
app.*.symbols
37+
38+
# Obfuscation related
39+
app.*.map.json
40+
41+
# Android Studio will place build artifacts here
42+
/android/app/debug
43+
/android/app/profile
44+
/android/app/release

‎packages/string_tokenizer/example/.metadata

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# This file tracks properties of this Flutter project.
2+
# Used by Flutter tool to assess capabilities and perform upgrades etc.
3+
#
4+
# This file should be version controlled.
5+
6+
version:
7+
revision: 2ad6cd72c040113b47ee9055e722606a490ef0da
8+
channel: stable
9+
10+
project_type: app
11+
12+
# Tracks metadata for the flutter migrate command
13+
migration:
14+
platforms:
15+
- platform: root
16+
create_revision: 2ad6cd72c040113b47ee9055e722606a490ef0da
17+
base_revision: 2ad6cd72c040113b47ee9055e722606a490ef0da
18+
- platform: ios
19+
create_revision: 2ad6cd72c040113b47ee9055e722606a490ef0da
20+
base_revision: 2ad6cd72c040113b47ee9055e722606a490ef0da
21+
22+
# User provided section
23+
24+
# List of Local paths (relative to this file) that should be
25+
# ignored by the migrate tool.
26+
#
27+
# Files that are not part of the templates will be ignored by default.
28+
unmanaged_files:
29+
- 'lib/main.dart'
30+
- 'ios/Runner.xcodeproj/project.pbxproj'

‎packages/string_tokenizer/example/README.md

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# ui_device_example
2+
3+
Demonstrates how to use the ui_device plugin.
4+
5+
## Getting Started
6+
7+
This project is a starting point for a Flutter application.
8+
9+
A few resources to get you started if this is your first Flutter project:
10+
11+
- [Lab: Write your first Flutter app](https://docs.flutter.dev/get-started/codelab)
12+
- [Cookbook: Useful Flutter samples](https://docs.flutter.dev/cookbook)
13+
14+
For help getting started with Flutter development, view the
15+
[online documentation](https://docs.flutter.dev/), which offers tutorials,
16+
samples, guidance on mobile development, and a full API reference.

‎packages/string_tokenizer/example/analysis_options.yaml

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# This file configures the analyzer, which statically analyzes Dart code to
2+
# check for errors, warnings, and lints.
3+
#
4+
# The issues identified by the analyzer are surfaced in the UI of Dart-enabled
5+
# IDEs (https://dart.dev/tools#ides-and-editors). The analyzer can also be
6+
# invoked from the command line by running `flutter analyze`.
7+
8+
# The following line activates a set of recommended lints for Flutter apps,
9+
# packages, and plugins designed to encourage good coding practices.
10+
include: package:flutter_lints/flutter.yaml
11+
12+
linter:
13+
# The lint rules applied to this project can be customized in the
14+
# section below to disable rules from the `package:flutter_lints/flutter.yaml`
15+
# included above or to enable additional rules. A list of all available lints
16+
# and their documentation is published at
17+
# https://dart-lang.github.io/linter/lints/index.html.
18+
#
19+
# Instead of disabling a lint rule for the entire project in the
20+
# section below, it can also be suppressed for a single line of code
21+
# or a specific dart file by using the `// ignore: name_of_lint` and
22+
# `// ignore_for_file: name_of_lint` syntax on the line or in the file
23+
# producing the lint.
24+
rules:
25+
# avoid_print: false # Uncomment to disable the `avoid_print` rule
26+
# prefer_single_quotes: true # Uncomment to enable the `prefer_single_quotes` rule
27+
28+
# Additional information about this file can be found at
29+
# https://dart.dev/guides/language/analysis-options

‎packages/string_tokenizer/example/ios/.gitignore

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
**/dgph
2+
*.mode1v3
3+
*.mode2v3
4+
*.moved-aside
5+
*.pbxuser
6+
*.perspectivev3
7+
**/*sync/
8+
.sconsign.dblite
9+
.tags*
10+
**/.vagrant/
11+
**/DerivedData/
12+
Icon?
13+
**/Pods/
14+
**/.symlinks/
15+
profile
16+
xcuserdata
17+
**/.generated/
18+
Flutter/App.framework
19+
Flutter/Flutter.framework
20+
Flutter/Flutter.podspec
21+
Flutter/Generated.xcconfig
22+
Flutter/ephemeral/
23+
Flutter/app.flx
24+
Flutter/app.zip
25+
Flutter/flutter_assets/
26+
Flutter/flutter_export_environment.sh
27+
ServiceDefinitions.json
28+
Runner/GeneratedPluginRegistrant.*
29+
30+
# Exceptions to above rules.
31+
!default.mode1v3
32+
!default.mode2v3
33+
!default.pbxuser
34+
!default.perspectivev3

‎packages/string_tokenizer/example/ios/Flutter/AppFrameworkInfo.plist

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
3+
<plist version="1.0">
4+
<dict>
5+
<key>CFBundleDevelopmentRegion</key>
6+
<string>en</string>
7+
<key>CFBundleExecutable</key>
8+
<string>App</string>
9+
<key>CFBundleIdentifier</key>
10+
<string>io.flutter.flutter.app</string>
11+
<key>CFBundleInfoDictionaryVersion</key>
12+
<string>6.0</string>
13+
<key>CFBundleName</key>
14+
<string>App</string>
15+
<key>CFBundlePackageType</key>
16+
<string>FMWK</string>
17+
<key>CFBundleShortVersionString</key>
18+
<string>1.0</string>
19+
<key>CFBundleSignature</key>
20+
<string>????</string>
21+
<key>CFBundleVersion</key>
22+
<string>1.0</string>
23+
<key>MinimumOSVersion</key>
24+
<string>11.0</string>
25+
</dict>
26+
</plist>

‎packages/string_tokenizer/example/ios/Flutter/Debug.xcconfig

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
#include "Generated.xcconfig"

‎packages/string_tokenizer/example/ios/Flutter/Release.xcconfig

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
#include "Generated.xcconfig"

0 commit comments

Comments
 (0)