Support for the weightedConfidence missing value strategy

[garasubin99]

Hi, I train a random forests model and it is generated into a pmml file. I used an AI platform to train it and it was evaluated normally at that environment. However, when I use JPMML to classify, different kinds of errors were thrown.

Like

org.jpmml.model.UnsupportedAttributeException: Attribute with value TreeModel@missingValueStrategy=weightedConfidence is not supported
	at org.jpmml.evaluator.tree.ComplexTreeModelEvaluator.handleMissingValue(ComplexTreeModelEvaluator.java:259)
	at org.jpmml.evaluator.tree.ComplexTreeModelEvaluator.handleTrue(ComplexTreeModelEvaluator.java:196)
	at org.jpmml.evaluator.tree.ComplexTreeModelEvaluator.handleTrue(ComplexTreeModelEvaluator.java:204)
	at org.jpmml.evaluator.tree.ComplexTreeModelEvaluator.handleTrue(ComplexTreeModelEvaluator.java:204)
	at org.jpmml.evaluator.tree.ComplexTreeModelEvaluator.handleTrue(ComplexTreeModelEvaluator.java:204)
	at org.jpmml.evaluator.tree.ComplexTreeModelEvaluator.evaluateTree(ComplexTreeModelEvaluator.java:147)
	at org.jpmml.evaluator.tree.ComplexTreeModelEvaluator.evaluateClassification(ComplexTreeModelEvaluator.java:112)
	at org.jpmml.evaluator.ModelEvaluator.evaluateInternal(ModelEvaluator.java:446)
	at org.jpmml.evaluator.mining.MiningModelEvaluator.evaluateSegmentation(MiningModelEvaluator.java:595)
	at org.jpmml.evaluator.mining.MiningModelEvaluator.evaluateClassification(MiningModelEvaluator.java:303)
	at org.jpmml.evaluator.ModelEvaluator.evaluateInternal(ModelEvaluator.java:446)
	at org.jpmml.evaluator.mining.MiningModelEvaluator.evaluateInternal(MiningModelEvaluator.java:224)
	at org.jpmml.evaluator.ModelEvaluator.evaluate(ModelEvaluator.java:300)

I use several sets of features involved in this pmml and it was evaluated normally. However, there are many unexpected input parameter characteristics that cause the model to throw this error. In other words, some features will be scored correctly, while others will be scored incorrectly. Please tell me how to solve this situation?

[garasubin99]

Is this phenomenon related to discrete features? Because some features of this random forest model are discrete, is it related to the fact that all values ​​cannot be obtained during training?

[vruusmann]

org.jpmml.model.UnsupportedAttributeException: Attribute with value TreeModel@missingValueStrategy=weightedConfidence is not supported

As the exception message clearly states, the TreeModel@missingValueStrategy attribute has been set to weightedConfidence value, which is not supported as of today (JPMML-Evaluator 1.7.1).

This attribute value has not been implemented, because I haven't come across any PMML files that use such as missing value handling strategy.

I used an AI platform to train it and it was evaluated normally at that environment.

What AI platform is that?

I'd be very much interested in seeing it, and any PMML documents exported from it.

Once I have a "proof" that the TreeModel@missingValueStrategy=weightedConfidence has real-life use cases, I'd be more than happy to implement its support into JPMML-Evaluator.

[vruusmann]

Is this phenomenon related to discrete features?

Definitely "No".

Is it related to the fact that all values ​​cannot be obtained during training?

Probably "Yes" - the weightedConfidence attribute value essentially orders the TreeModel evaluation engine to switch to an alternative scoring algorithm if the default scoring algorithm cannot be used because of missing feature values.

Again, a sample PMML model with a sample sparse CSV input would be very helpful here.

[garasubin99]

Thank you a lot for your answers! My pmml file has several segments, in which are organized as follows:

I wonder that how can I modify the weightedConfidence strategy to score in jpmml environment?

[vruusmann]

Repeating my earlier inquiry - what was the software that was used for training these random forest models? Does this software have a config option for toggling the "missing value handling" strategy? Perhaps there's a built-in option to change it from weightedConfidence to something else?

I'm personally aware of any open-source model training software that supports such a combination.

I wonder that how can I modify the weightedConfidence strategy to score in jpmml environment?

You can edit the TreeModel@missingValueStrategy attribute value as needed.

The six possible values are listed here.

The default value is none. If all your Node elements have a Node@score attribute defined, then I would recommending setting the TreeModel@missingValueStrategy attribute to lastPrediction, which basically means "return the last node's score" (the parent node of this non-scorable node).

[vruusmann]

@garasubin99 No reproducible example(s) means no fix.

Can you use your AI Platform to train a RF classifier for the iris dataset, export it into a PMML document, and attach it here?

[garasubin99]

It's very kind of you to answer my questions! That really helps and inspired me a lot! The AI ​​platform I used cannot be disclosed because it is confidential to my school. So I am very sorry that this platform cannot be disclosed. I will take your advice and give it a try. Words fail me when I want to express my thanks!