Release queue

From DPWiki
Jump to: navigation, search
DP Official Documentation - Content Providing and Project Management


General Info: Queues and Release Processing

A release queue (also queue) is a holding area for Distributed Proofreaders projects to be released into the rounds for proofing or formatting.

Each round has its own set of queues which enable Distributed Proofreaders to have a wide variety of projects always available in each round.

Every 15 minutes the system runs a release script (Automodify) which checks each round individually: P1, P2, P3, F1, F2.

For each round it processes, Automodify first determines whether there are projects that have finished the round, or which need to have pages reclaimed, and takes the appropriate actions. After that, Automodify examines the round's queues in order, looking for the next queue which is enabled for processing. "Disabled" queues are ignored. For each enabled queue the script decides whether it should make any projects Available for processing in the round, as described below.

The Automodify process produces a log each time it is run. The logs for approximately the current month are in viewable form here for anyone interested in them. See below for more details of the process.

A queue is a FIFO...

Each release queue is defined by a set of conditions that determine whether a project will be included in the queue or not. These include:

A single project can be in multiple queues and enter the round from any of those queues.

Each queue also has a set of release conditions, which are rules of the form "If there are fewer than N projects in this round of this type, release a new project" or "If there are fewer than N pages in projects in this round of this type, release a new project."

Within a queue, projects are listed in the order that they entered the queue. It's a first in, first out system. You can choose a release queue to view from the Release Queues main page.

There is no guarantee that a given queue will ever release anything: all of its books may release via other means and keep enough books in the round that the release conditions are never met.

... except when it is not

Once a project reaches the head of a queue, additional tests are performed and these may block its release (in which case the project behind it will be checked)

  • There can only be one project for each author in each round. Note: Author names are compared exactly as specified in the project. Alan Smith is a different author from Alan A. Smith and from Alan Smith, Jr.
  • A single PM can have no more than 13 projects in each round
  • Each invocation of the release script will normally release at most one project per PM per round.

Exceptions:

  • The authors Anonymous, Various, and Unknown can each have up to four projects in each round.
  • Projects defined with a Special Day code (except "Children's Book Week" or "Type In Project") are exempt from the author and PM limits regarding the number of projects in each round.
  • One objective for each enabled queue is to have at least one page available for proofreading in its round at all times. To accomplish that, when the project-release processing finds that there are no pages available in a round for projects that are in that queue, but there were projects that could have been released except for the author or PM limits, the first one of those projects in the queue will be allowed to run. This "emergency release valve" means that the PM or author whose project is chosen will end up with at least one more project active than would normally be allowed.

Flowchart and Detailed Example

For a Flowchart of the Project Release process, see Project Release Flowchart.

For an example run of the Automodify script, see Automodify log example for queues.

General Examples

The P1 release rule for English beginners only projects says that a new project should release if there are fewer than 5 beginners only project pages available. That could be 4 pages in one project or 3 pages in one project and 1 in another. Only the total matters. If the next project in the beginners only project queue has the same author as a project (of any sort or language) that is already in P1, then it won't release and the system will check the second project in the queue. And so on, until it either determines that nothing can release, or it finds something that can.

Most queue release conditions are defined in terms of the number of projects, rather than the number of pages. So, for example, the P1 English periodical queue will release something when there are fewer than 5 English periodicals in P1. A few are defined in terms of both pages and number of projects. Most of the P1 LOTE queues will release something when there are either fewer than 4 projects or fewer than 400 pages available.

Different queues for different rounds

In P1, P2 and P3 there are many release queues, each of which works as described above.

In F1, there are three queues, one for English, one for Languages other than English, and one for the DP Anniversary. Projects move into F1 fairly quickly once they finish P3. F2 has PM queues in addition to the three that F1 has. Neither formatting round has special queues other than the anniversary queue, or any of the genre, language, or special day queues.

PM Queues

A PM Queue is a certain type of release queue which helps ensure that a minimum of one project is available in a given round for a specific Project Manager. PM queues are available for P1, P2, P3, and F2.

PMs can request that PM Queue(s) be set up for themselves by contacting db-req.


Genre Queues

Genre Queues exist in P1, P2 and P3. There are different genre queues for English, French, German, Dutch, Spanish and Italian. Most genre queues represent several different genres. For example, the English Reference queue can handle English projects that are categorized as Bibliography, Dictionary, Encyclopedia or Reference.

Click here for a list of the current Genre Queues in P1, P2, and P3.


Special Day queues and processing

When do Special Day queues start?

All special day queues start at the server time of 8am of the day before the "open" date for that queue. For example the St. Valentine's Day special day queue starts at the server time of 8am 13 February.

Handling Special Day queues

Special Day projects are handled differently in several ways.

  • For P1:
    • A project marked with a special day code won't release until the day (no matter the genre) but it will release on that day, subject to the limits in the queue definition which can restrict the number of projects active at the same time. The special day processing allows PMs to prep books ahead of time but be sure that they won't release until the designated time. If any projects remain in the queue at the end of the day, a PF or Squirrel will usually push them into P1 manually.
    • Because we are a global community, special "days" are defined to start when that date first comes along anywhere in the world (effectively when New Zealand hits that date) and to close when the last of our community ends the day (effectively at midnight in Hawaii). So each "day" runs for well over 24 hrs. That is why you may see projects releasing into P1 at a time that appears to be a day before or a day after the designated date.
  • For P2, projects can release either through the genre queues or through the special day queues, which ever comes up first. That is, in P1, books marked for a special day are held until that day arrives. In P2, they are not held back, but rather are pushed forward if a special day happens along.
  • P3 handles special day projects similarly to P2, except that the special day queues release only one book at a time.
  • We currently have no Special Day queues defined and active in F1 and F2, and projects release only through the normal queues defined in those rounds.
  • As mentioned above, the usual limits based on the number of projects an author or PM may have in a round are bypassed for projects marked with a special day code (except "Children's Book Week" or "Type In Project").

How can I help get a project released?

Because of the way that various queues can interact, it is hard for anyone to tell exactly when a given project is likely to release. If you want to hurry things along for a project that is waiting, your best bet is to work on other projects in the same genre, if the project is in English or French, or in the same language for other LOTE. (But bear in mind the author and PM restrictions.)

The release queues are funny beasts that can interact in some very odd ways. Every feature of the queues has been implemented for what seemed a logical reason, but the subsequent interaction of those features can be confusing and can make it hard to predict how projects will progress through the queues.

Will this change?

Definitely! The queue system is actively managed, which means that detailed points in this article may become inaccurate (please let us know if that happens).

The queue definitions can be modified by the site administrators (Squirrels). Sometimes we will modify the code to change the way that the queues are handled. Major changes to the queues are announced in the forums, usually in Updated Queue Tweaks and possibly in Recent Additions to the Site.

To comment or request edits to this page, please contact Monicas wicked stepmother or DACSoft.

Return to DP Official Documentation Menu