Support for getting token consumptions for each query

[SolidLao]

Hi,

I believe it would be very useful that FlockMTL could support getting token consumptions, including prompt tokens and completion tokens, for each query.

It is practical to monitor the token consumptions, and then the money costs for each query that uses LLMs.

Thanks.

[queryproc]

Hello @SolidLao! Thank you for the feedback.

We will add this as a logging mechanism that can be configured (ON/OFF) very soon. It is a top priority in our roadmap.

[SolidLao]

Thanks so much! Excited to see this feature soon!