Issue With MarkUpMatches() Removing Tags

[brychart]

Hello, I'm using Bunit to test some pages and before it has worked great but for some reason I'm running into this issue.

I create a TestContext and render my page. If I put find th in its own variable I can see it contains "<th>Title</th>". My test line is cut.Find("th").MarkupMatches("<th>Title</th>"); however when I run it I get this error:

[egil]

Uhm that's weird.

What version of bUnit are you using and what version dotnet?

[brychart]

I'm on Dotnet Core 3.1, and Bunit is v1.0.0 - beta 8.

It's very odd, I've used this exact syntax with Bunit on this exact project earlier and it worked fine. The only difference is we're using components from the MatBlazor library so I had to mock some parts of the TestContext:

    `public TestContext SetUp()
    {
        var context = new TestContext();
        context.Services.AddMockJSRuntime();
        context.Services.AddSingleton<HttpClient>();
        context.Services.AddSingleton<NavigationManager, TestNavManager>();

        return context;
   }`

The navigation manager is an empty class:

public class TestNavManager : NavigationManager { protected override void NavigateToCore(string uri, bool forceLoad) { // Do nothing. } }

[egil]

It should definitely not do that. Can you try to change the markup such that it is a td or h1 instead of a th, just to see if this has any effect.

[brychart]

I did a few more tests and it looks like the issue happens with td, th and tr but not h1 or p.

[egil]

Thanks. That is weird. I'll investigate a bit and get back to you.

[egil]

It looks like the root of this problem is with AngelSharp, at least the way bUnit is using it under the hood. AngleSharp throws away the table related elements away and just keeps text content.

The following test fail:

using Xunit;
using AngleSharp;
using AngleSharp.Html.Parser;
using System.Threading.Tasks;

namespace AngleSharpTableTagParsing
{
  public class UnitTest1
  {

    [Theory]
    [InlineData("h1")] // passes
    [InlineData("p")] // passes
    [InlineData("table")] // fails
    [InlineData("thead")] // fails
    [InlineData("tbody")] // fails
    [InlineData("th")] // fails
    [InlineData("td")] // fails
    [InlineData("tr")] // fails
    public async Task ParsingTagsCorrectly(string tag)
    {
      var config = Configuration.Default.WithCss();
      var context = BrowsingContext.New(config);
      var parser = context.GetService<IHtmlParser>();
      var document = await context.OpenNewAsync();
      var markup = $"<{tag}>Title</{tag}>";

      var nodes = parser.ParseFragment(markup, document.Body);

      Assert.Equal(AngleSharp.Dom.NodeType.Element, nodes[0].NodeType);
    }
  }
}

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="AngleSharp" Version="0.14.0" />
    <PackageReference Include="AngleSharp.Css" Version="0.14.2" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.6.1" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="1.3.0">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

</Project>

Am I using AngleSharp wrong in this case, @FlorianRappl?

[egil]

Same happens with <TargetFramework>netcoreapp3.1</TargetFramework>

[egil]

Forcing the runtime to sdk 3.1.x instead of 5.0.preview.8 by adding a global.json with this inside:

{
  "sdk": {
    "version": "3.1"
  }
}

Issue still shows.

[FlorianRappl]

Thanks.

This is expected by AngleSharp as the W3C HTML5 spec parses based on context. Without a context (i.e., just using <body>) you cannot work with table elements in a non table.

A source code like <table>test</table> is anyway "invalid" and will be changed by the parser to test<table></table>. Likewise, without a table context <td>test</td> becomes test.

You can test these things easily, e.g., in https://www.w3schools.com/html/tryit.asp?filename=tryhtml_basic

<!DOCTYPE html>
<html>
<body>
<script>
document.body.innerHTML = '<table><td>hello</td></table>';
alert(document.body.innerHTML);
</script>
</body>
</html>

vs

<!DOCTYPE html>
<html>
<body>
<script>
document.body.innerHTML = '<td>hello</td>';
alert(document.body.innerHTML);
</script>
</body>
</html>

vs

<!DOCTYPE html>
<html>
<body>
<script>
document.body.innerHTML = ''; // just for nicer output
var table = document.body.appendChild(document.createElement('table'));
table.innerHTML = '<td>hello</td>';
alert(document.body.innerHTML);
</script>
</body>
</html>

So you'd essentially need to either detect or allow setting the context. If somebody wants to have just "td" then it must be set to a table context.

From the perspective of AngleSharp / the parser everything is "as expected". Now what that means for bUnit we still need to figure out. To be concluded.

[egil]

Thanks @FlorianRappl for helping clearing this up. This is sort of a "works as unexpected" 🤷‍♂️😉

I will add an issue to see if I can get a good solution for this for the special cases around TR, TD, TH, TBODY and THEAD. There might be other elements that the HTML parser does not expect in the BODY context. So I will have to investigate further.

But for now, @brychart, you will have to assert a different way than using MarkupMatches. Maybe use cut.Find("th").TextContent.Should.Be("Title").

[brychart]

Appreciate it! I'll try that. I also found that .ToMarkup() includes the <th> and <td> tags but just with \r and \n which were trivial to strip out.

[egil]

Appreciate it! I'll try that. I also found that .ToMarkup() includes the <th> and <td> tags but just with \r and \n which were trivial to strip out.

That surprises me actually. @FlorianRappl can you explain that?

Here is the method for reference:

		public static string ToMarkup(this IMarkupFormattable markupFormattable)
		{
			if (markupFormattable is null)
				throw new ArgumentNullException(nameof(markupFormattable));

			using var sw = new StringWriter();
			var formatter = new PrettyMarkupFormatter()
			{
				NewLine = Environment.NewLine,
				Indentation = "  "
			};
			markupFormattable.ToHtml(sw, formatter);
			return sw.ToString();
		}

Do AngleSharp keep the <th> and <td> tags around as strings, even if it doesn't parse it into a elements?

[FlorianRappl]

Well as discussed I don't really agree to

works as unexpected

because "expected" is a quite subjective term. Somebody familiar with the W3C spec would say "works as expected". As this is the reference and describes how all browsers / HTML works, I'd say the ones who don't know this are to blame. It may not be known so much, but its still something that is in the spec.

As to the question regarding ToMarkup - there is no magic here. Not sure what exactly is done here using ToHtml, but it only prints the DOM.

For instance,

using System;
using AngleSharp;
					
public class Program
{
	public static async Task Main()
	{
		var document = await BrowsingContext.New(Configuration.Default).OpenNewAsync();
		document.Body.InnerHtml = "<td>test/<td>";
		Console.WriteLine(document.ToHtml());
	}
}

will result in

<html><head></head><body>test</body></html>

So no td here.

[egil]

Thanks.

Ps. The works as unexpected was targeted at me, not AngleSharp. I didn't expect it to behave like that 🤗