.NET 6 WebApplicationFactory test throws System.InvalidOperationException : The entry point exited without ever building an IHost.

[mike-novatech]

Description
When I have Serilog configured on my Program.cs and if I run more than one test that uses WebApplicationFactory I get an exception thrown System.InvalidOperationException : The entry point exited without ever building an IHost. accessing the application factory services.

Reproduction
Program.cs

using System.Text.Json.Serialization;
using Serilog;

Log.Logger = new LoggerConfiguration()
    .WriteTo.Console()
    .CreateBootstrapLogger();

Log.Information("Starting up...");

try
{
    var builder = WebApplication.CreateBuilder(args);
    builder.Host.UseSerilog((ctx, lc) => lc.ReadFrom.Configuration(ctx.Configuration));

    var configuration = builder.Configuration;
    var services = builder.Services;
    // Add services to the container.

    services.AddControllers()
         .AddJsonOptions(options =>
         {
             options.JsonSerializerOptions.Converters.Add(new JsonStringEnumConverter());
         })
         .AddControllersAsServices();
    services.AddEndpointsApiExplorer();

    services.AddSwaggerGen();

    var app = builder.Build();

    app.UseSerilogRequestLogging();

    app.UseCors(builder =>
        builder
            .WithOrigins(configuration.GetSection("AllowedOrigins")
                .AsEnumerable()
                .Select(kvp => kvp.Value)
                .Where(origin => !string.IsNullOrEmpty(origin))
                .ToArray()
            )
            .SetIsOriginAllowedToAllowWildcardSubdomains()
            .AllowAnyMethod()
            .AllowAnyHeader()
            .AllowCredentials());

    // Configure the HTTP request pipeline.
    if (app.Environment.IsDevelopment())
    {
        app.UseSwagger();
        app.UseSwaggerUI();
    }

    app.UseHttpsRedirection();

    app.UseAuthentication();
    app.UseAuthorization();

    app.MapControllers();

    app.Run();
    return 0;
}
catch (Exception ex)
{
    Log.Fatal(ex, "Host terminated unexpectedly.");
    return 1;
}
finally
{
    Log.CloseAndFlush();
}

#pragma warning disable CA1050 // Declare types in namespaces
public partial class Program { } // so you can reference it in tests
#pragma warning restore CA1050 // Declare types in namespaces

appsettings.json:

{
  "AllowedHosts": "*",
  "Serilog": {
    "AllowedHosts": "*",
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Warning",
        "Microsoft.Hosting.Lifetime": "Information"
      }
    },
    "Enrich": [
      "FromLogContext"
    ],
    "Filter": [
      {
        "Name": "ByExcluding",
        "Args": {
          "expression": "@mt = 'An unhandled exception has occurred while executing the request.'"
        }
      }
    ],
    "WriteTo": [
      {
        "Name": "Console",
        "Args": {
          "outputTemplate": "[{Timestamp:HH:mm:ss} [{Level}]: {Message:l}{NewLine}{Exception}"
        }
      }
    ]
  }
}

TestApplication.cs:

using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.Hosting;

namespace Api.Tests;

internal class TestApplication : WebApplicationFactory<Program>
{
    private readonly string environment;

    public TestApplication(string environment = "Development")
    {
        this.environment = environment;
    }

    protected override IHost CreateHost(IHostBuilder builder)
    {
        builder.UseEnvironment(environment);
        return base.CreateHost(builder);
    }
}

SwaggerTest.cs:

using System.Linq;
using System.Threading.Tasks;
using FluentAssertions;
using Microsoft.Extensions.DependencyInjection;
using Swashbuckle.AspNetCore.Swagger;
using Xunit;

namespace Api.Tests.Swagger;

public class SwaggerTest
{
    [Fact]
    public async Task OperationIds_Should_Be_Unique()
    {
        await using var application = new TestApplication();
        var swaggerProvider = application.Services.GetRequiredService<ISwaggerProvider>();
        var swagger = swaggerProvider.GetSwagger("v1");
        var operationIds = swagger.Paths.Values.SelectMany(path => path.Operations.Values.Select(operation => operation.OperationId)).ToList();

        operationIds.Should().OnlyHaveUniqueItems();
    }

    [Fact]
    public async Task OperationIds_Should_Be_Unique2()
    {
        await using var application = new TestApplication();
        var swaggerProvider = application.Services.GetRequiredService<ISwaggerProvider>();
        var swagger = swaggerProvider.GetSwagger("v1");
        var operationIds = swagger.Paths.Values.SelectMany(path => path.Operations.Values.Select(operation => operation.OperationId)).ToList();

        operationIds.Should().OnlyHaveUniqueItems();
    }
}

Expected behavior
Tests pass, and does not throw calling application.Services

Relevant package, tooling and runtime versions

<PackageReference Include="Serilog.AspNetCore" Version="4.1.0" />
<PackageReference Include="Serilog.Expressions" Version="3.2.1" />

dotnet --version:

6.0.102

Additional context
The tests pass if I comment out the line builder.Host.UseSerilog((ctx, lc) => lc.ReadFrom.Configuration(ctx.Configuration)); from Program.cs.

[pikoscielniak]

I have the same. Really annoying.

[nblumhardt]

Thanks for the report! Someone will need to set up "break on all exceptions", or step through the test in a debugger to narrow it down. I assembled a test project using the sample above, without Swashbuckle and my test can instantiate the test host and resolve ILogger<Anything> successfully.

(Another approach might be to try dropping the Swashbuckle-related config from Program.cs and see whether you still hit an issue.)

[pikoscielniak]

After removing Swashbuckle from the project issue still occurs.

[pikoscielniak]

Stack trace looks like this:

  System.InvalidOperationException : The entry point exited without ever building an IHost.
 Stack trace:
     at Microsoft.Extensions.Hosting.HostFactoryResolver.HostingListener.CreateHost()
   at Microsoft.Extensions.Hosting.HostFactoryResolver.<>c__DisplayClass10_0.<ResolveHostFactory>b__0(String[] args)
   at Microsoft.AspNetCore.Mvc.Testing.DeferredHostBuilder.Build()
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateHost(IHostBuilder builder)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.DelegatedWebApplicationFactory.CreateHost(IHostBuilder builder)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.ConfigureHostBuilder(IHostBuilder hostBuilder)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureServer()
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(DelegatingHandler[] handlers)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(Uri baseAddress, DelegatingHandler[] handlers)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateClient(WebApplicationFactoryClientOptions options)
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateClient()

[nblumhardt]

Thanks @pikoscielniak; does "Break on all exceptions" show anything up?

[pikoscielniak]

It's really strange. When I run tests in debug all tests pass.

[pikoscielniak]

The tests also pass when I run them with coverage.

[pikoscielniak]

Probably I know why tests pass in debug or in coverage because than they are executed by IDE one after another, but problem only happens when tests are run parallel.

[pikoscielniak]

I can confirm that the problem happens only when tests are run in parallel. When I run test with the config:

{
  "$schema": "https://xunit.net/schema/current/xunit.runner.schema.json",
  "parallelizeTestCollections": false
}

they all pass, but unfortunately it's not a solution at my case. I strongly need run tests parallelly.

[nblumhardt]

Does the issue persist if you switch from CreateBootstrapLogger() to CreateLogger()?

[pikoscielniak]

Changing Serilog initialization in Program.cs from:

Log.Logger = new LoggerConfiguration()
    .WriteTo.Console()
    .CreateBootstrapLogger();

to:

Log.Logger = new LoggerConfiguration()
    .WriteTo.Console()
    .CreateLogger();

solved the problem.
@nblumhardt could you explain why is that?

[nblumhardt]

@pikoscielniak thanks for helping to track it down. CreateBootstrapLogger() creates a ReloadableLogger, which the UseSerilog() callback reconfigures and then freezes.

Freezing is a stateful operation:

https://github.com/serilog/serilog-extensions-hosting/blob/dev/src/Serilog.Extensions.Hosting/Extensions/Hosting/ReloadableLogger.cs#L80

Not sure how best to avoid this in testing.. 🤔

[pikoscielniak]

@nblumhardt does using CreateLogger instead of CreateBootstrapLogger have any dangerous influence on running on production?

[nblumhardt]

https://nblumhardt.com/2020/10/bootstrap-logger/#why-cant-we-just-do-both has some info on the problems CreateBootstrapLogger() resolves; if those issues don't apply to you then it's fine to just use CreateLogger() instead.