Skip to content

Conversation

@mcarmonaa
Copy link
Contributor

@mcarmonaa mcarmonaa commented Apr 11, 2018

This PR covers the second part from #242, having the producer a flag to requeue jobs from the buried queue.

Right now in the buried queue should be stay only temporary-errors and fatal errors, so temporary errors are requeued and fatal errors are left on the buried queue.

Note that those jobs in the buried queue stayed as pending in the database, so there's no need to change their status when they are requeed.

There are two points to think about:

  • Maybe we should drop the fatal errors, tag them with some header, or set them in the database with some special status.
  • The --republish-jobs flag works as an addition to the producer regular job, so if we republish jobs from the buried queue and we give the producer a list of repositories where there are repositories that they are in the buried queue too, we will have duplicated jobs on the queue. An option could be have a command just to republish jobs.

UPDATE: after talk with @ajnavarro this functionality should work as a service that only do the requeue task.

What do you think @src-d/data-retrieval?

This PR depends on src-d/framework#40

… buried queue

Signed-off-by: Manuel Carmona <manu.carmona90@gmail.com>
@mcarmonaa mcarmonaa changed the title Add logic to the producer to be able to requeue failed jobs [WIP] Add logic to the producer to be able to requeue failed jobs Apr 11, 2018
This was referenced Apr 11, 2018
@mcarmonaa
Copy link
Contributor Author

Superseded by #253

@mcarmonaa mcarmonaa closed this Apr 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant