ValueTask, pooling and async overhead

A price cache answers most lookups from memory, yet through an async Task<int> method each of those answers still creates an object on the heap: 72 bytes a call, measured below. At 1 million lookups a second that is about 72 MB of garbage a second for prices that were already there.

You will measure what an async method allocates, return ValueTask<T> where the result is often ready at once, use it in the ways that are safe, and decide from numbers when this async overhead is worth any change at all.

An async method that finishes at once still allocates a task

alloc.cs calls a price lookup 100,000 times in each of five ways and reads the garbage collector’s own count of the bytes the calls allocated:

// Bytes allocated per call by an async price lookup, measured with the GC's own counters.
const int Calls = 100_000;
var prices = new Dictionary<string, int> { ["ABC"] = 41250, ["LOW"] = 7 };

await Task.Yield(); // from here on Main runs on a pool thread, and its own state machine is already on the heap
long total = 0;
for (int i = 0; i < 1_000; i++) // warm up every path once, so that the measured loops allocate only their calls
{
    total += await FromCacheTaskAsync("ABC") + await FromCacheValueTaskAsync("ABC") + await LoadTaskAsync() + await LoadValueTaskAsync();
}

long start = GC.GetAllocatedBytesForCurrentThread();
for (int i = 0; i < Calls; i++) total += await FromCacheTaskAsync("ABC");
Report("Task<int>, the price is in the cache", GC.GetAllocatedBytesForCurrentThread() - start);

start = GC.GetAllocatedBytesForCurrentThread();
for (int i = 0; i < Calls; i++) total += await FromCacheTaskAsync("LOW");
Report("Task<int>, the cached value is 7", GC.GetAllocatedBytesForCurrentThread() - start);

start = GC.GetAllocatedBytesForCurrentThread();
for (int i = 0; i < Calls; i++) total += await FromCacheValueTaskAsync("ABC");
Report("ValueTask<int>, the price is in the cache", GC.GetAllocatedBytesForCurrentThread() - start);

// A call that really waits resumes on another thread, so it is counted with the process-wide counter.
start = GC.GetTotalAllocatedBytes(precise: true);
for (int i = 0; i < Calls; i++) total += await LoadTaskAsync();
Report("Task<int>, the call waits", GC.GetTotalAllocatedBytes(precise: true) - start);

start = GC.GetTotalAllocatedBytes(precise: true);
for (int i = 0; i < Calls; i++) total += await LoadValueTaskAsync();
Report("ValueTask<int>, the call waits", GC.GetTotalAllocatedBytes(precise: true) - start);

Console.WriteLine($"(sum of every price read: {total})");

async Task<int> FromCacheTaskAsync(string code)
{
    if (prices.TryGetValue(code, out int price)) return price;
    await Task.Yield();
    return 0;
}

async ValueTask<int> FromCacheValueTaskAsync(string code)
{
    if (prices.TryGetValue(code, out int price)) return price;
    await Task.Yield();
    return 0;
}

static async Task<int> LoadTaskAsync()
{
    await Task.Yield(); // stands in for the database: the method stops and resumes on a pool thread
    return 41250;
}

static async ValueTask<int> LoadValueTaskAsync()
{
    await Task.Yield();
    return 41250;
}

// Bytes per call, to the nearest byte: a background thread of the runtime may add a few hundred bytes to a run.
static void Report(string what, long bytes) => Console.WriteLine($"{what,-42} {Math.Round(bytes / (double)Calls),4} bytes a call");

Task<int>, the price is in the cache         72 bytes a call
Task<int>, the cached value is 7              0 bytes a call
ValueTask<int>, the price is in the cache     0 bytes a call
Task<int>, the call waits                    96 bytes a call
ValueTask<int>, the call waits              104 bytes a call
(sum of every price read: 16665700000)

A hit through async Task<int> cost 72 bytes a call, a hit through async ValueTask<int> cost nothing, and the cached value 7 cost nothing even as a Task<int>. A call that really waited allocated about 100 bytes whichever type it returned. Step through where those bytes come from:

Task<int> is a class, so a result that is ready at once still needs a new task object to carry it, unless the runtime keeps a finished task for that value, as it did for 7. A ValueTask<int> is a struct that can hold the int itself, so the answer travels without touching the heap. When a method really waits, it must keep its local variables until it resumes, so its state machine moves to the heap either way. That is the whole case for ValueTask in one line: it saves the heap allocationheap allocation: Creating an object on the managed heap, which the garbage collector must later find and free. Every new class instance, array and boxed value is one, and so is each new Task<T>; a struct held in a local variable is not. of calls that finish at once, and nothing else.

A ValueTask holds a result or a source, and is awaited once

A ValueTask<T>ValueTask<T>: A struct an async method can return instead of Task<T>. It holds the result itself when the result is ready at once, so that path allocates nothing, or else a Task<T> or a reusable source; it may be consumed only once. wraps one of three things: the result itself, a Task<T>, or a source object that will produce the result. Because that object may be reused, a ValueTask may be consumed only once. consume.cs shows the three safe ways, on a cache that loads a missing price from a database:

using System.Collections.Concurrent;

// The three safe ways to consume a ValueTask<int>, on a cache that loads a missing price once.
var cache = new PriceCache();

// 1. Await it directly, once: the usual way, and the right one almost everywhere.
int abc = await cache.GetAsync("ABC");
Console.WriteLine($"1. await: ABC {abc}");

// 2. Read Result only after IsCompletedSuccessfully said it is ready; otherwise await it. Either way it is consumed
//    once, and nothing touches it afterwards.
ValueTask<int> pending = cache.GetAsync("ABC");
if (pending.IsCompletedSuccessfully)
{
    Console.WriteLine($"2. complete at once, read with Result: ABC {pending.Result}");
}
else
{
    Console.WriteLine($"2. not complete yet, awaited: ABC {await pending}");
}

// 3. Need a Task, for WhenAll, to keep it, or to await it twice? Convert each one once with AsTask.
Task<int>[] both = [cache.GetAsync("XYZ").AsTask(), cache.GetAsync("PQR").AsTask()];
int[] two = await Task.WhenAll(both);
Console.WriteLine($"3. AsTask and WhenAll: XYZ {two[0]}, PQR {two[1]}");
Console.WriteLine($"loads from the database: {cache.Loads}");

sealed class PriceCache
{
    private readonly ConcurrentDictionary<string, int> prices = new();

    private int loads;

    public int Loads => loads;

    public async ValueTask<int> GetAsync(string code)
    {
        if (prices.TryGetValue(code, out int price)) return price; // a hit: the ValueTask holds the int itself
        price = await LoadAsync(code);
        prices[code] = price;
        return price;
    }

    private async Task<int> LoadAsync(string code)
    {
        Interlocked.Increment(ref loads);
        await Task.Delay(20); // the database call
        return code switch { "ABC" => 41250, "XYZ" => 9900, _ => 15000 };
    }
}

1. await: ABC 41250
2. complete at once, read with Result: ABC 41250
3. AsTask and WhenAll: XYZ 9900, PQR 15000
loads from the database: 3

The usual way is to await the call directly, once. A hot loop may check IsCompletedSuccessfully and read Result only when it says yes, awaiting otherwise, which is how the second line got 41250 without an await. When a task is needed, to pass to Task.WhenAll, to keep or to await twice, call AsTask() once and use the task; the third line combined two loads that way, and the database was called three times in all.

Pooling goes further. A ValueTask may wrap an IValueTaskSource, an object a library reuses for one operation after another, as sockets and channels do, and the PoolingAsyncValueTaskMethodBuilder, switched on per method with an attribute, reuses the state machines of finished calls. Both kinds of pooling make the once-only rule matter more, because a reused object answers for the next operation, not yours.

Misusing a ValueTask, reproduced on a reused source Pro: opens with a Pro, Premium or Ultimate pass

When async overhead matters, measured at a 95 % hit ratio Pro: opens with a Pro, Premium or Ultimate pass

A lab: a price cache whose hits allocate nothing Pro: opens with a Pro, Premium or Ultimate pass

Practise

The self-check covers the first two sections: what an async method allocates, and how a ValueTask is consumed. Your weakest idea comes up first, and every answer tells you why it holds or misses:

Interview questions

Fresher · When should a method return ValueTask<T> instead of Task<T>?

When the method usually has its result at once, is called often enough for the allocation of a new task per call to show in measurements, and its callers simply await it. A cache lookup that hits most of the time is the typical case: returned as a struct, a ready result costs no heap allocation, while Task<T> allocates for most values. When the method really waits, both types allocate about the same, so ValueTask gains nothing there. The price is a stricter contract: a ValueTask may be awaited only once and never by two callers, so code that keeps it or passes it to Task.WhenAll must call AsTask(). Microsoft’s guidance is to return Task<T> by default.

Key takeaways

• An async Task<T> method that finishes at once still allocates a task for most results, 72 bytes a call here, while ValueTask<T> carries a ready result without allocating.
• A method that really waits moves its state machine to the heap whatever it returns, so ValueTask saves only the calls that complete at once.
• Await a ValueTask once and directly, read Result only after IsCompletedSuccessfully, and call AsTask() once when a task is needed; return Task<T> unless measurements say otherwise.