[Feature Request] Regarding the versions of a suite and individual testcases

[olw-medcom]

Hey there,

We use the metadata fields 'gitb:version' in both testsuites and testcases. However I am unable to find it represented in the UI. I would love for it to be visible in the generated reports and in the historical test sessions overview.

Furthermore it would be absolutely lovely if a small tag such as 'new version released since your last execution' was indicated, so that our vendors are notified of a new version?

Do you think that would be possible?

If my feature request is unclear, please tell me and I shall provide some examples of what I mean

[costas80]

Hi @olw-medcom.

We use the metadata fields 'gitb:version' in both testsuites and testcases. However I am unable to find it represented in the UI. I would love for it to be visible in the generated reports and in the historical test sessions overview.

The topic of versioning on test suites and test cases has come up often in the past to be honest. As it stands the gitb:version attribute is used as internal metadata (it's displayed to administrators), whereas if you want to display version information to users, you do this currently by adding the version in the test suite/case name. This is also due to how the test session history currently works, where executed test sessions are like a snapshot in time and capture the name of the test suite/case at the time of execution (including the version number).

Adding a dedicated version number indicator at this point would only serve the visual representation (e.g. like a tag on the test suite/case), given that you never have multiple versions of the same test suite/case active at the same time. I would suggest you follow what most projects do by just adding a version suffix to the name of test suites/cases to have it displayed both on the UI and produced reports. Would this work for you?

Furthermore it would be absolutely lovely if a small tag such as 'new version released since your last execution' was indicated, so that our vendors are notified of a new version?

How new versions are communicated similarly varies greatly between projects. Also, even in your case I would assume that this notification is not something you would always want. For example:

• Imagine that you extend your current test cases, improving things like their logging, documentation and user interactions (which are not important conformance-wise). In this case you may have new versions for test cases but don't want to alert users to them if they already finished their testing.
• In case you make more significant changes, then you already have the option of invalidating previous tests. As such, a version update would result in a 100% completed conformance statement becoming a 90% completed one, because 10 out of 100 tests have significant changes that need re-execution.

It seems that the indicator of a new version being available for completed tests is more pertinent in the first scenario above, where you are keeping the previously completed tests but want to highlight that they were completed with an earlier test suite version. If this is indeed the case, then the feature becomes a bit niche again as it would apply when making changes that don't affect the testing history (i.e., minor updates). I'm saying this knowing that for most projects using the ITB, once users complete their tests they would not revisit them unless they need to to complete their testing.

Having said all this however, we had in mind to introduce a more general purpose feature that could suit your needs. The idea is to define community notifications that appear to users when they connect. The community admin will be able to define notifications at different levels (info, warn, error), that are persistent until acknowledged, and that are defined as community-wide or even per organisation. You can achieve something similar today with the landing pages (community default and organisation-specific ones) but this is much heavier as a feature. The notifications I'm referring to would display in the same way you see the alert popups in the bottom-right corner, with the difference that they can be persistent (like the service health warnings the Test Bed admin gets). Given this feature, you could address what you're going for by making your test suite update and then creating a community-wide, persistent, info-level alert that states that there is a new test suite version available.

What do you think of this approach?

[olw-medcom]

Hey @costas80, you have some good points and indeed it would be quite a challenge to make a versioning and notification system that would work for all.
I will therefore take the suffix/prefix approach and indeed community notifications would be a massively appreciated feature! An added bonus would be the ability to target specific organizations even.

As a side note. I presented your platform to my team and we have decided to go ahead and use the test bed as a test management system and I foresee me sending quite many feature requests related to this in this year. I'm looking forward to our continued collaboration, let me know if you ever need something from me 😊

[costas80]

community notifications would be a massively appreciated feature!

Nice! I'll bump it in our backlog and make sure it gets added to the next release. I'll let you know when it's ready to use in the nightly builds.

we have decided to go ahead and use the test bed as a test management system

Great news! Happy to see that the ITB works well for you guys.

I foresee me sending quite many feature requests related to this in this year

Keep them coming! We love getting good feature requests and ideas from users in the field. We can never think ourselves of all the things that come up with different projects. Also, as the ITB is general purpose, it is used in hugely different setups and use cases, so user feedback from all these diverse scenarios is invaluable.

I'm looking forward to our continued collaboration, let me know if you ever need something from me

Me too. In the future we could make a publication about your use case in our news stream (that also gets promoted by our comms team). We have no shortage of articles from EU/International/National public administrations and organisations, but not so much from Industry. Yours seems to be a nice use case which would make for an interesting piece. I'll ping you in the not so distant future about this if you don't mind :)

[costas80]

In the meantime, let's keep this issue open until the notification feature is available in the nightly builds.

[olw-medcom]

That's great, thank you!

I don't mind at all, please do ping me! 😊

[olw-medcom]

Two follow up questions:

1. What is the gitb:version tag used for? Can I omit it in my xml files?
2. Is there a way to mange the name of the specification through xml? Currently I can put a suffix on my testsuite but not the specification

[costas80]

What is the gitb:version tag used for? Can I omit it in my xml files?

The test suite version currently is displayed on the UI to administrators when viewing the test suite's details. The test case version is not displayed or otherwise used. In both cases, the TDL specs require the version string to be defined so you can't simply omit it. We could potentially make an update to the specs to make it optional however.

Is there a way to mange the name of the specification through xml?

No, because specifications are not covered in the TDL specs and are thus not defined through test suites. The specification name can be set either through the UI or via the REST API.

[olw-medcom]

That makes sense, thanks costas!

By the way - I have tried to omit it and everything seems to be working fine?

[costas80]

By the way - I have tried to omit it and everything seems to be working fine?

Yes, my bad! This has been optional for a while now but the TDL docs (that I referred to) missed the update.