(Open-Ended) Features Accounting for Conversational Context

[xehu]

The vast majority of our features are applied at the level of a single message. However, as we are learning from the process of hand-labeling conversations, sometimes we need to be able to look at the overall context of a conversation in order to fully understand what a message means.

This is a relatively open-ended issue. The basic premise is that we need to think through the kinds of features that might need to be processed in the context of a larger conversation --- what does that look like?

One answer might be in some of our existing features: for example, "forward flow" (and related features) measures messages in the context of how similar they are to what's already been said. The design for this feature might be somewhere along those lines.

---

• Copy the Template. Go to the Feature Wiki and Copy/Paste the Feature Template into a new page.

• Fill out the Template. Fill out the basic information for the feature in the template. Use the template to document your plan for implementation and major design decisions; if anything changes along the way, update the documentation as you go.

• Create a new Feature file. Create a new file in the folder feature_engine/features. The name of the file should be NAME_features.py, where NAME is the name of your feature.

• Code your feature. Write the code for the feature.

• Evaluate/Unit Test. Come up with a method of evaluating your feature. How do you know that the feature is 'correct?' For simple features, this may be trivial; for more complex ones, you may need to break down each sub-function and unit test them separately, or validate on external data. Add your unit tests to feature_engine/test_featurize.py.