Another String type for best performance and memory management

[okarpov]

From my point of view (and within a project i was involved into) .NET/C# definitely needs another String type which can be used to optimize performance and constant memory growing when working with a lot of strings and Networking.

My observation is the following - Http/TCP/Socket working with string types do consume A LOT of memory/heap especially in highly intensive communication apps (parsing/replacing/substringing urls, HttpHeaders, etc).

Having another type of String object should always be the reference type which may be passed into framework functions and got as returning values of framework functions.

Replace, Substring, etc. consume a lot of memory creating new string objects in applications which highly intensive use strings and networking communication.

[danmoseley]

How would this new kind of string be stored and what specifically would be different that would make it more efficient in those cases?

[okarpov]

How would this new kind of string be stored and what specifically would be different that would make it more efficient in those cases?

It needs to be passed by ref all the time and use byte[] behind to not copy string and to not increase memory with every string manipulation.

[Clockwork-Muse]

Note, too, that many of the more modern versions of these APIs use Memory/Span specifically to avoid string allocation because of this problem.

[okarpov]

main "gap" as i understand is HttpClient.SendAsync/HttpRequestMessage

[Clockwork-Muse]

You can use a streamed content body for HttpRequestMessage (although I'm not finding a simple example to give you).
That aside, normally you're going to want to use one of the more specific method operations (eg, PostAsync, which are a little better.

If you're worried about headers, cache an initial set. You have to duplicate the list/dictionary that goes out the door (Assuming you have a per-request header), but most of the individual header keys/values will be constant, and can thus be the same reference object.

[okarpov]

we use SendAsync, so how to cache Http Headers here?
Thank you

[Clockwork-Muse]

At minimum, simply iterating over the original, and adding (probably via one of the non-validating methods) will do it, simply because that means the key/value will be the original reference object. There's also a memberwise clone, which might do it as well (I haven't reviewed it enough).

[okarpov]

oh, yes, we do it. just thought may be there is another way for that.
thank you anyway

[davidfowl]

dotnet/corefxlab#2350 - Get some popcorn.

[okarpov]

yeah, this should be good improvement

[okarpov]

but seems was archived on corefx and i did not find any evidence of migration to dotnet/runtime, did you?

[Joe4evr]

@okarpov It's not migrated because the consensus is "It's not happening."

[huoyaoyuan]

Replace, Substring, etc. consume a lot of memory creating new string objects in applications which highly intensive use strings and networking communication.

Also note that if you take a small substring from a huge string in non-copy way and store it for long use, the huge string will be kept alive, which can hurt memory usage too. The tradeoff isn't that simple.

[okarpov]

Replace, Substring, etc. consume a lot of memory creating new string objects in applications which highly intensive use strings and networking communication.

Also note that if you take a small substring from a huge string in non-copy way and store it for long use, the huge string will be kept alive, which can hurt memory usage too. The tradeoff isn't that simple.

yes, also the XmlSerializer consumes a lot of memory because of intensive string manipulations behind when you need to process a lot of XMLs even if they all are deserialized into same single type using singleton (type,root) constructor.