Roadmap to 1.0?

We use BenchmarkDotNet in an FDA-regulated environment to design and develop a medical device. We have to account for all of the off-the-shelf (OTS) components. We are responsible for understanding the development process, maintenance and support lifecycle, design limitations, and other quality properties of every OTS components we use.

The lack of a major version and a roadmap does not prevent use of a component, but it introduces a documentation burden to explain or justify the use of a prerelease component.

It would be awesome to see GitHub releases and milestones, at least, and maybe a "roadmap" project.

[AnthonyMastrean]

I posted this issue from another account that I meant to delete... I have sync'ed my email addresses 😳😅

Ah, I see that GitHub moved the milestones into the Issues interface. I see that every version has a changelog entry on the website, but not a GitHub "release" which provides a useful, albeit hidden, Atom feed link (this is useful for our configuration management practice).

[AndreyAkinshin]

I see that every version has a changelog entry on the website, but not a GitHub "release" which provides a useful, albeit hidden, Atom feed link (this is useful for our configuration management practice).

It's a valid point, I have created a separate issue for that: #2036. I will think about ways to automate the pushing of changelogs to the GitHub releases (I don't want to do it manually each time, we are trying to completely automate the release process which is half-manual right now).

Roadmap to 1.0

I'm not completely satisfied with our statistical engine. While it's quite mature and stable, there are some corner cases that may cause problems in measurements and result presentations. I think that 1.0 should be released once the new statistical engine is ready. Unfortunately, it's a huge research project without ETA (you can find research notes in my blog).

Regardless of the fact that 1.0 is not published yet, the current BenchmarkDotNet version is stable and widely-adopted. I believe that the lack of the official 1.0 version shouldn't a blocker to adopt BenchmarkDotNet in your project.

[AnthonyMastrean]

It's all good! I just need to provide more thorough documentation. The guides on the website are actually very useful in providing evidence of quality (good practices, how it works, etc.).

https://benchmarkdotnet.org/articles/guides/how-it-works.html

[timcassell]

@AndreyAkinshin Before 1.0, I was thinking some of the public types that are only public for the generated projects to consume could be made internal, and add [InternalsVisibleTo] so the generated projects can still use them. What do you think?

[Edit] I'm not sure if that would work for the InProcessEmitToolchain, but I think so.

[YegorStepanov]

Please, do not silently release 1.0 without an announcement.

Topics that I want to cover during the year:

• BDN should not throw unhandled exceptions.
• Unify toolchains. (for example, InProcessEmitToolchain supports [Params("111")] public int Field;)
• Move different checks to validators (to simplify code duplication with analyzers).
• Add analyzers.
• Try to replace RPlots to the C# plots (ScottPlot 5 is the only C# library that fits BDN, I haven't experimented with it yet. I'll write a prototype).
• Rewrite the documentation: from the sample-based docs to the normal :)

[filzrev]

Redesigning the command-line arguments is also a candidate before releasing version 1.0.
See: #3162 (comment)