Skip to content

Jpleger/sqs #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
jeanpatrickleger wants to merge 47 commits into
base: release/0.1.0
Choose a base branch
from
Open

Jpleger/sqs #30

jeanpatrickleger wants to merge 47 commits into from

Conversation

@jeanpatrickleger
Copy link
Contributor

@jeanpatrickleger jeanpatrickleger commented Jun 14, 2022

Fixes # .

Description

This is a pull request for AWS SQS ( Simple Queue Service ) support. This proposes an alternative option to using RMQ. The documentation about the activation of the plugin and its configuration can be found here : https://github.com/Project-MONAI/monai-deploy-messaging/blob/jpleger/SQS/src/Messaging/SQS/README.MD

Status

Ready/Work in progress/Hold

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • All tests passed locally by running ./src/run-tests-in-docker.sh.
  • Documentation comments included/updated.
  • User guide updated.
  • I have updated the changelog

mocsharp and others added 14 commits April 5, 2022 16:45
@mocsharp
Update release job
2bb7035
@mocsharp
Skip publishing duplicate nuget packages
367882e
@mocsharp
Requires manual trigger to deploy to NuGet.org
05376c0
@mocsharp
Update ci.yml
a852ec7 
Publish & close release only on main branch.
@mocsharp
Update release job
da1c02b
@mocsharp
Skip publishing duplicate nuget packages
bfafff0
@mocsharp
Requires manual trigger to deploy to NuGet.org
529d9c5
@mocsharp
Change target framework to .net standard 2.1
5fff089
@mocsharp
Merge branch 'release/0.1.0' into vchang/netstandard
d1450c8
@mocsharp
Merge pull request #2 from Project-MONAI/vchang/netstandard
776b217 
Target .NET Standard 2.1
@mocsharp
Update GitVersion.yml
f90bf2e
@mocsharp
Update GitVersion.yml
649c656
@mocsharp
Update ci.yml
fd4e2f6
@mocsharp
[Breaking] Unit tests for RabbitMQ plugin (#4)
ffcf29a 
* Implement unit tests for RabbitMQ plugin
* Fix sonar code smells
* Update PR template

Signed-off-by: Victor Chang <vicchang@nvidia.com>
@codecov
Copy link

codecov bot commented Jun 14, 2022 •
edited
Loading

Codecov Report

Merging #30 (ed2f2b5) into release/0.1.0 (ffcf29a) will decrease coverage by 27.11111%.
The diff coverage is 53.31412%.

Impacted file tree graph

@@                   Coverage Diff                    @@
##           release/0.1.0         #30          +/-   ##
========================================================
- Coverage       93.77778%   66.66667%   -27.11112%     
========================================================
  Files                 19          36          +17     
  Lines                450        1065         +615     
  Branches              15          70          +55     
========================================================
+ Hits                 422         710         +288     
- Misses                24         343         +319     
- Partials               4          12           +8
Flag Coverage Δ
unittests 66.66667% <53.31412%> (-27.11112%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/Messaging/Common/Log.cs 100.00000% <ø> (ø)
.../Messaging/Configuration/ConfigurationException.cs 50.00000% <ø> (+25.00000%) ⬆️
src/Messaging/Messages/MessageBase.cs 100.00000% <ø> (ø)
...rc/Messaging/RabbitMq/RabbitMqConnectionFactory.cs 0.00000% <0.00000%> (ø)
src/Messaging/SQS/ConfigurationKeys.cs 0.00000% <0.00000%> (ø)
src/Messaging/SQS/Log.cs 0.00000% <0.00000%> (ø)
src/Messaging/SQS/QueueFormatter.cs 0.00000% <0.00000%> (ø)
src/Messaging/SQS/SQSMessagePublisherService.cs 0.00000% <0.00000%> (ø)
src/Messaging/SQS/SQSMessageSubscriberService.cs 0.00000% <0.00000%> (ø)
src/Messaging/Common/MessageConversionException.cs 25.00000% <25.00000%> (ø)
... and 30 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffcf29a...ed2f2b5. Read the comment docs.

mocsharp and others added 15 commits June 14, 2022 09:41
@mocsharp @jeanpatrickleger
Event validation & add Task related events (#7)
367ce0f 
* Rename Message to Event used in the message body.
* Add validation to events and validate event messages on all properties recursively.

Signed-off-by: Victor Chang <vicchang@nvidia.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jackschofield23 @jeanpatrickleger
Add subscription no requeue reject (#9)
5a314d3 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@mocsharp @jeanpatrickleger
New RunnerCompleteEvent (#10)
e3c4c91 
- Add RunnerCompleteEvent
- Rename TaskCompleteEvent to TaskUpdateEvent with new fields and changes.
- Allow multiple inputs/outputs for TaskDispatchEvent
- Add Accepted TaskStatus

Signed-off-by: Victor Chang <vicchang@nvidia.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@mocsharp @jeanpatrickleger
Add additional failure reasons (#11)
7fe7ce0 
Signed-off-by: Victor Chang <vicchang@nvidia.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@mocsharp @jeanpatrickleger
Update FailureReason.cs
2402d20 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@mocsharp @jeanpatrickleger
Ability to convert Message to JsonMessage<T> (#13)
bfc08d9 
* Ability to convert Message to JsonMessage<T>

Signed-off-by: Victor Chang <vicchang@nvidia.com>

* Add SubscribeAsync with a Func<> callback

Signed-off-by: Victor Chang <vicchang@nvidia.com>

* Update unit test for SubscribeAsync

Signed-off-by: Victor Chang <vicchang@nvidia.com>

* Use existing RabbitMQ connection if available, otherwise, create a new one.

Signed-off-by: Victor Chang <vicchang@nvidia.com>

* Fix code smells

Signed-off-by: Victor Chang <vicchang@nvidia.com>

* Breaking: IRabbitMqConnectionFactory to CreateChannel and grantee a single connection is used

Signed-off-by: Victor Chang <vicchang@nvidia.com>

* APIs to bind multiple topics to a single queue

Signed-off-by: Victor Chang <vicchang@nvidia.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@mocsharp @jeanpatrickleger
Handle corrupted messages (#12)
7909d03 
* Rename RunnerCompleteEvent to TaskCallbackEvent with additional Metadata property.
* Handle corrupted messages
* Check message properties
* Use RabbitMQ Timestamp
* Declare exchanges are durable & !auto delete
* Fix JsonProperty mapping
* Include metadata in the TaskUpdateEvent

Signed-off-by: Victor Chang <vicchang@nvidia.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@mocsharp @jeanpatrickleger
Update TaskUpdateEvent.cs
19b141d 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jackschofield23 @jeanpatrickleger
update task dispatch event
6436c66 
Signed-off-by: jackschofield23 <jack.schofield@answerdigital.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@mocsharp @jeanpatrickleger
Make credentials optional in TaskDispatchEvent.Storage (#17)
50c66fc 
Signed-off-by: Victor Chang <vicchang@nvidia.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jackschofield23 @jeanpatrickleger
change task status name
bdb5be7 
Signed-off-by: jackschofield23 <jack.schofield@answerdigital.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jackschofield23 @jeanpatrickleger
Rename workflow instance id (#20)
f637151 
Signed-off-by: jackschofield23 <jack.schofield@answerdigital.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@mocsharp @jeanpatrickleger
Intermediate Storage for TaskDispatchEvent (#21)
5f360a7 
* Add intermediate storage for TaskDispatchEvent
* Implement ICloneable for Storage

Signed-off-by: Victor Chang <vicchang@nvidia.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@mocsharp @jeanpatrickleger
Throw MessageConversionException on error converting JSON event messa…
64437ab 
…ge (#22)

Signed-off-by: Victor Chang <vicchang@nvidia.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jackschofield23 @jeanpatrickleger
update taskupdate to include output artifacts
310d005 
Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
jackschofield23 and others added 10 commits June 14, 2022 09:41
@jackschofield23 @jeanpatrickleger
update task callback event (#24)
1761aad 
Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jackschofield23 @jeanpatrickleger
fix task callback and task update event (#25)
8f63ccd 
Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jackschofield23 @jeanpatrickleger
add payload id to task dispatch (#26)
b076491 
Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jackschofield23 @jeanpatrickleger
add export status (#28)
c167be8 
Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger @awsjpleger
Options to enable secure RabbitMQ (#27)
16a53b7 
* RMQ connection support for SSL/TLS (no certificate check).
* Refactored RabbitMqConnectionFactory to support TLS connection and Port number config.
* Update on IRabbitMqConnectionFactory Summary.

Co-authored-by: JP Leger <jpleger@amazon.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
Added SQS plug-in as RMQ alternative.
d0ba49e 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
SQS Support adde to messaging lib
bc71ebd 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
Code analysis fixes.
4edabf0 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@awsjpleger @jeanpatrickleger
remove unecessary folder.
8cdb394 
Signed-off-by: JP Leger <jpleger@amazon.com>
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
problematic README.MD removed
75973f0 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
README.MD added back.
fad266c 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
Code smells corrections.
4d930e8 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
Code smells corrections.
fa737a0 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
Code smells corrections.
9fe3d1b 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
Code smells corrections.
674029f 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
Code smells corrections.
47781e1 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
Code smells corrections.
2e5ab53 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@jeanpatrickleger
Code smells corrections.
ed2f2b5 
Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>
@mocsharp mocsharp force-pushed the release/0.1.0 branch 2 times, most recently from e08d9a2 to 6b0d721 Compare June 27, 2022 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mocsharp mocsharp Awaiting requested review from mocsharp

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@jeanpatrickleger @mocsharp @jackschofield23 @awsjpleger