Knowledge Sharing Session

Polly

The .NET Resilience Library

Resilience Engineering Fault Tolerance

Session Overview

What to Expect

We'll cover the core concepts of resilience engineering, the different Polly policies, and practical techniques for building fault-tolerant .NET applications.

Topic

Polly — The .NET Resilience Library

Format

Technical session + live demo + Q&A

Introduction to Polly and the resilience challenges it solves
Understanding retry, circuit breaker, timeout, bulkhead isolation, and fallback policies
Live implementation and hands-on demo of adding Polly policies to a .NET application
Best practices for combining policies (policy wrapping) and integrating with HttpClientFactory
Interactive Q&A and discussion

Introduction

What is Polly & Why Do We Need It?

Polly is a free .NET code library that automatically handles small, temporary failures for you — the little "glitches" that happen when your app talks to something else over a network, like a database or another website.

🌍 Real-World Example

Think about calling a friend on the phone. If the call drops because of a bad signal, you don't give up — you simply redial. Polly does exactly this for your code: if a call to a payment API fails because of a brief network glitch, Polly automatically "redials" it a few times before giving up, so the user never notices anything went wrong.

Without Polly

App

→

Exception

→

User gets error

With Polly

App

→

Retry

→

Success

The user never even notices.

Instead of crashing, Polly lets you:

Retry, break failing connections, set timeouts, provide fallbacks, limit concurrency, and combine multiple strategies together.

An intelligent protection layer for:

REST & Payment APIs, SQL Server, Redis, RabbitMQ, Azure/AWS services, and any network or file-system call.

Core Philosophy

What Does "Resilience" Mean?

In software engineering, resilience means a system can keep working even when something goes wrong — recovering quickly instead of crashing or stopping completely.

Fault Tolerance

The bridge is built so strong it can handle strong winds without any damage.

Resilience

The bridge can bend during a big storm, absorb the impact, and return to normal instead of collapsing.

💡 Core Philosophy

"In distributed networks, failures are inevitable. Software shouldn't crash because of them; it should survive them." Resilient systems show cached data, retry after a short delay, temporarily disable only the affected feature, and keep everything else running normally.

Evolution & Architecture

Polly Architecture & System Design

Application

→

Resilience Pipeline

Retry · Timeout · Circuit Breaker
Rate Limiter · Hedging · Fallback

→

External Service

Old Polly (v7)

Each rule (retry, timeout, circuit breaker) was its own separate "Policy" object. Combining several required extra wrapping — slower and heavier on memory.

New Polly (v8)

A ResiliencePipeline is a single, unified "assembly line" — one faster, lighter-weight pipeline replacing all the separate Policy objects.

Prevents Cascading Failures

If Service B slows down, the circuit breaker cuts the connection so Service A doesn't exhaust its own resources waiting.

Decoupled Fallbacks

Instead of a raw 500 error, a failed recommendations engine can fall back to a cached, generic product list.

Protects Resources

Polly prevents a single user (tenant) from crashing an entire application by consuming all its network resources.

Getting Started

Installing & First Example

Terminal

dotnet add package Polly
# For HTTP integration:
dotnet add package Microsoft.Extensions.Http.Resilience
      

FirstPipeline.cs

var pipeline = new ResiliencePipelineBuilder()
    .AddRetry(new RetryStrategyOptions())
    .Build();

await pipeline.ExecuteAsync(async token =>
{
    return await client.GetAsync(url, token);
});
      

Most Common Strategy

Retry Strategy

If an operation fails, don't give up immediately — wait a moment and try it again, because the problem might already be gone.

🌍 Real-World Example

A delivery driver knocks on your door. No answer — maybe you were in another room. Instead of leaving forever, they try again in a few minutes. Retry works the same way for a failed network call.

Fail

→

Retry

→

Retry

→

Success

Execution Example

Attempt 1 → 500 Error

Attempt 2 → 503 Error

Attempt 3 → 200 OK

Retry.cs

var pipeline = new ResiliencePipelineBuilder()
    .AddRetry(new RetryStrategyOptions
    {
        MaxRetryAttempts = 3
    })
    .Build();
        

Optimizing Retries

Tuning Retries: Backoff, Jitter & Predicates

1. Fixed Delay

Always waits the same amount of time.

2s → 2s → 2s

2. Exponential Backoff

Increases delay after each failure. Reduces server pressure.

1s → 2s → 4s → 8s

3. Jitter

Prevents 1000 clients from retrying at exactly the same time (Thundering Herd).

1.2s → 2.8s → 4.5s

🔧 Retry Only What Makes Sense

Don't retry 400 Bad Request or 401 Unauthorized — retrying an error that will never go away wastes time and can trigger account lockouts. A predicate tells Polly exactly which exceptions or status codes are worth retrying, by exception type or by HTTP status code.

TunedRetry.cs

.AddRetry(new RetryStrategyOptions
{
    MaxRetryAttempts = 4,
    Delay = TimeSpan.FromSeconds(1),
    BackoffType = DelayBackoffType.Exponential,
    UseJitter = true,
    ShouldHandle = new PredicateBuilder()
        .Handle<HttpRequestException>()
        .Handle<TimeoutException>()
})
      

Protection

Timeout Strategy

Give an operation a maximum amount of time to finish. If it hasn't responded by then, stop waiting and treat it as failed — instead of waiting forever.

🌍 Real-World Example

You call a customer support line and get put on hold. You don't stay on the line for 3 hours — you decide "if no one picks up in 10 minutes, I'll hang up and try another way." A Timeout does exactly this for a slow API call.

Suppose API hangs forever.

Waiting... Waiting... Waiting...

Polly stops it. Prevents hung requests and frees resources.

Timeout.cs

var pipeline = new ResiliencePipelineBuilder()
    .AddTimeout(TimeSpan.FromSeconds(10))
    .Build();
        

Fail Fast

Circuit Breaker

Named after the electrical circuit breaker in your house. If a service keeps failing, stop sending it requests for a while so it can recover.

🌍 Real-World Example

When too many appliances draw power at once, the breaker in your fuse box "trips" to prevent a fire. You wait, fix the problem, then reset it. Polly's Circuit Breaker does the same for a failing API.

Closed

Current flows normally. Requests pass through — everything is healthy.

Open

Tripped. All requests are blocked instantly and fail fast, giving the downstream service time to recover.

Half-Open

After cool-down, one "test" request is let through. Success → closes again. Failure → trips back open.

CircuitBreaker.cs

.AddCircuitBreaker(new CircuitBreakerStrategyOptions
{
    FailureRatio = 0.5, // 50% failure rate
    MinimumThroughput = 10,
    BreakDuration = TimeSpan.FromSeconds(30)
})
      

Graceful Degradation

Fallback Strategy

Have a "Plan B" answer ready. If every attempt to get the real result fails, give the user something useful instead of an error screen.

🌍 Real-World Example

A restaurant is out of your first-choice dish. Instead of sending you away hungry, the waiter suggests a similar dish that's available. A Fallback does this for software — if live data can't be fetched, it serves cached or default data instead.

Suppose API fails entirely.

Instead of throwing an exception, return:

Cached data (e.g. weather forecast)
A default message
A secondary payment provider

Fallback.cs

.AddFallback(new FallbackStrategyOptions<HttpResponseMessage>
{
    FallbackAction = args =>
    {
        return Outcome.FromResultAsValueTask(
            new HttpResponseMessage(HttpStatusCode.OK)
            {
                Content = new StringContent("Cached Data")
            });
    }
})
        

Traffic Control & Low Latency

Rate & Concurrency Limiters, and Hedging

Rate Limiter

Controls requests per time period.

Example: server allows 100 requests/minute — Polly ensures you don't exceed the quota.

Concurrency Limiter

Controls simultaneous requests — the v8 successor to Bulkhead Isolation.

Example: only 10 requests at once; extra requests wait.

Hedging (New in v8)

Sends a backup request when the original is slow — the fastest response wins.

Trade-off: uses more resources; best for read-only or geo-distributed services.

The Real Power

Combining Strategies & HTTP Integration

A single call can pass through several protective layers, like security checkpoints at an airport — each layer only steps in if the ones before it couldn't fix the problem.

Timeout

→

Retry

→

Circuit Breaker

→

Fallback

→

External Service

Program.cs — ASP.NET Core

builder.Services
    .AddHttpClient("Weather")
    .AddResilienceHandler("standard", builder =>
    {
        builder
            .AddRetry(new RetryStrategyOptions())
            .AddTimeout(TimeSpan.FromSeconds(5))
            .AddCircuitBreaker(new CircuitBreakerStrategyOptions())
            .AddFallback(...);
    });
      

Now every request from this client automatically flows through the full Polly pipeline.

Summary

Strategy Cheat Sheet & Best Practices

Strategy	Use When...	Real-World Analogy
Retry	Temporary failures like timeouts or 503 responses	Redialing a dropped phone call
Circuit Breaker	A downstream service is repeatedly failing	An electrical breaker tripping during a surge
Timeout	Operations might hang indefinitely	Hanging up after 10 minutes on hold
Fallback	A default response or cached data is acceptable	Waiter suggesting an alternate dish
Hedging	Low latency is critical; duplicate requests are OK	Booking two rideshares, keeping whichever arrives first
Rate / Concurrency Limiter	You must respect API limits or protect resources	A bouncer letting in 5 people per minute

✅ Best Practices

Retry only transient failures, with exponential backoff + jitter.
Log retry attempts and circuit state changes.
Combine retries with timeouts & breakers.
Use idempotency keys for retried POST requests.

❌ Common Mistakes

Infinite retries, or retrying 401/403/400 errors.
Long timeout values without a CancellationToken.
Applying the same retry policy to every operation.

Real-World Deep Dive

Case Studies from Production

The same strategies protect very different systems, at very different scales.

The Thundering Herd

A 3-second DB blip caused 50,000 requests to retry at once, crashing the database. Fix: exponential backoff + jitter smoothed retries across time.

The Gateway Collapse

A bank API slowed to 45s, exhausting all app threads. Fix: Timeout (2s) + Circuit Breaker (60s open) + Fallback to an offline queue.

Microsoft's Cloud Standard

Every team writing custom retry code caused fragmentation. Fix: Microsoft embedded Polly v8 as a first-class citizen across Azure SDKs.

User → ASP.NET Core API → Resilience Pipeline (Retry · Timeout · Circuit Breaker · Fallback) → SQL Server / Payment API / Redis

Every downstream call is protected the same consistent way — one shared set of Polly rules.

Thank You

Polly — The .NET Resilience Library

Production-Ready Battle-Tested .NET Foundation Member

Since v5.0 in October 2016, Polly has been governed as a community-driven open-source project under the .NET Foundation — the same organization stewarding core .NET infrastructure.

Official Docs: pollydocs.org

GitHub: App-vNext/Polly