The testing guide for Active Job currently implies that when you queue a job it will be performed.
This isn't true; by default jobs are enqueued, not performed.
This PR fleshes out the docs a bit to show both examples, and adds a test to confirm the default behaviour.
Also add some additional words to make it clear that the modules also
implement handling the exceptions configured with rescue_from, because
it was not immediately clear that happened without reading the code.
Currently, we don't document the use case for ActiveJob's `queue_with_priority` block arguments. It seems necessary to document them in the API docs as well considering how useful this option is.
In commit 72300f97 the code for the `String` condition was simplified, and it's
now the same as the code for the `*PERMITTED_TYPES` condition, and `String` is
one of the `PERMITTED_TYPES` so we can just eliminate this condition and let
`*PERMITTED_TYPES` cover it.
This has utility for gems/modules included on jobs, which can tie into this behaviour and run something when a job fails.
after_discard respects the existing retry behaviour, but will run even if a retried exception is handled in a block.
In this test case we only check if the third child's queue adpter is nil or not. We should instead check if the queue adapter is the same as base adapter to ensure that it did not change when we changed qdapters for other jobs.
50f0c0f064/activejob/test/cases/queue_adapter_test.rb (L44-L52)
This gives queue adapters more freedom to name and organize their code.
For example, if `FancyQueue` wants to have their adapter at
`FancyQueue::ActiveJobAdapter`, the name would be `active_job` before this
change. After this change, they can implement `queue_adapter_name` to return
`fancy_queue`.
Before this change, setting a module as queue adapter would result in
`queue_adapter_name` being `"module"`. This PR fixes the name extraction logic
to handle module or class queue adapters.
Most of the other frameworks use a h1(`=`) instead of h2(`==`) for
class/module documentation. Having a h1 will improve SEO and makes
things look more consistent.
This also adds a missing title and escapes namespaces so it won't be
linked.
Currently when opening the main framework pages there is no introduction
to the framework. Instead we only see a whole lot of modules and the
`gem_version` and `version` methods.
By including the READMEs using the `:include:` directive each frameworks
has a nice introduction.
For markdown READMEs we need to add the :markup: directive.
[ci-skip]
Co-authored-by: zzak <zzakscott@gmail.com>
The display_name method is used by delayed job to log information
about a certain job, including failure messages. Whenever a job class
is moved or deleted, the instances still scheduled cannot be
constantized anymore, causing display_name and hence the log method to
raise an exception. In certain cases, e.g. when logging happens in a
rescue block, this may terminate the entire delayed job worker. With
the failsafe method, the worker handles failed jobs gracefully and
continues work, all with appropriate log output.
This file was introduced in de28930 because the Delayed Job JobWrapper
includes an adapter specific method that logs job parameters. Since it
was the first adapter specific test file, there was not previously any
logic to only run some files for each adapter, so this file was included
when running any adapter's tests.
This commit excludes the delayed_job specific test file from all of the
other adapter test tasks so that they do not require delayed_job and run
these tests.
de28930: de28930d46
* Remove Copyright years
* Basecamp is now 37signals... again
Co-authored-by: David Heinemeier Hansson <dhh@hey.com>
---------
Co-authored-by: David Heinemeier Hansson <dhh@hey.com>
The initial deprecation was introduced in 6.1:
bbfab0b33a
The configuration stayed with a deprecation in 7.0 but was not
preserving the behavior: 10bd5e59c3
This removes the configuration and deprecation.
Sidekiq has a useful optimisation called `push_bulk` that enqueues many jobs at
once, eliminating the repeated Redis roundtrips. However, this feature is not
exposed through Active Job, so it only works for `Sidekiq::Worker` jobs. This
adds a barrier to Active Job adoption for apps that rely on this feature. It
also makes it harder for other queue adapters to implement similar
functionality, as they then have to take care of serialization, callbacks, etc.
themselves.
This commit adds `ActiveJob.perform_all_later(<job1>, <job2>)`, backed by
Sidekiq's `push_bulk` and with a fallback to enqueuing serially if the queue
adapter does not support bulk enqueue.
The performance benefit for 1000 jobs can be more than an order of magnitude:
| Enqueue type | Serial time (ms) | Bulk time (ms) | Speedup |
| ------------------ | ---------------- | -------------- | ------- |
| Raw Sidekiq | 2661 | 119 | 22x |
| Active Job Sidekiq | 2853 | 208 | 14x |
(Measured in a simple test app in our production environment.)
Instrumentation for perform_all_later uses a new event `enqueue_all.active_job`
Since Sidekiq 7.0 requires Ruby 2.7, older versions of Rails that
support older Rubies are still tested against Sidekiq 6.x.
Sidekiq::MAJOR was added in 7.0 so it can't be used for version testing,
see 862dc5b
The config change is due to changes in Sidekiq 6.5. These were accounted
for in 7a069dc but removed in 6d31993.
Sidekiq::MAJOR was added in mperham/sidekiq@b4092e3, which is only
included in 7.0.0+ so we can't use it to check the version for
Sidekiq 6.
Since the test code is written to support both, the condition should
also support both.
Since engine initializers run later in the process, we need to run this
initializer earlier than the default.
This ensures they're all registered before the environments are loaded.
This commit adds `ActiveJob.deprecator` and replaces all usages of
`ActiveSupport::Deprecation.warn` in `activejob/lib` with
`ActiveJob.deprecator`.
Additionally, this commit adds `ActiveJob.deprecator` to
`Rails.application.deprecators` so that it can be configured via
settings such as `config.active_support.report_deprecations`.
This commit also removes a defunct `ActiveSupport::Deprecation.silence`
call that was added in 9eb4b4ed01 but not
removed when the deprecation was completed in
10bd5e59c3.
Fixes https://github.com/rails/rails/issues/46103
An issue exists if you set `config.active_record.query_log_tags` to an array that includes `:controller`, `:action`, or `:job`; the relevant item will get duplicated in the log line. This occured because the relevant railties would add the item to `config.active_record.query_log_tags` again during setup. This PR fixes that by only adding those items to the config if they aren't already set.
The issue proposed more documentation to work around this, but I think it's a bug and should be fixed directly.
I noticed a difference in behavior when inspecting `ActiveJob::Core#scheduled_at` in non-test environments vs when inspecting in while executing tests.
Prior to this change, `ActiveJob::TestHelper` would set `ActiveJob::Core#scheduled_at` to a `Time` object in tests,
however this attribute was set to a `Float` objects in non-test environments:
<d1aa6af5cc/activejob/lib/active_job/core.rb (L161)>
This change ensures parity in all runtime environments.
We recently let a few very easy to avoid warnings get merged.
The root cause is that locally the test suite doesn't run in
verbose mode unless you explictly pass `-w`.
On CI warnings are enabled, but there is no reason to look at the
build output unless something is failing. And even if one wanted
to do that, that would be particularly work intensive since warnings
may be specific to a Ruby version etc.
Because of this I believe we should:
- Always run the test suite with warnings enabled.
- Raise an error if a warning is unexpected.
We've been using this pattern for a long time at Shopify both in private
and public repositories.
The various LogSubscriber subclasses tend to subscribe to events
but then end up doing nothing if the log level is high enough.
But even if we end up not logging, we have to go through the
entire notification path, record timing etc.
By allowing subscribers to dynamically bail out early, we can
save a lot of work if all subscribers are silenced.
Previously configs of the form `config.active_job.X` were only forwarded to
`ActiveJob::Base`. This teaches Active Job to set them on `ActiveJob` directly
instead, if the setter exists.
For consistency, this more or less mirrors the way that Active Record does it.
Co-authored-by: Adrianna Chang <adrianna.chang@shopify.com>
Co-authored-by: Sam Bostock <sam.bostock@shopify.com>
---
Fix use_big_decimal_serializer Rails 7.1 default
This config should be enabled for new Rails 7.1 apps, or apps that have updated
their config to `load_defaults 7.1`, not disabled.
This also clarifies the config accessor comment.
---
Add contributor documentation comment to load_defaults
The process for introducing a change in behavior in Rails can be confusing to
new contributors, so a comment is added roughly explaining how to do so, and
what belongs in `load_defaults` and `new_framework_defaults`.
This comment is aimed at contributors, not consumers, so it is added within the
method, rather than above it.
Print default parent class for controller, job, and model generators.
Before:
[--parent=PARENT] # The parent class for the generated job
After:
[--parent=PARENT] # The parent class for the generated job
# Default: ApplicationJob
Previously, BigDecimal was listed as not needing a serializer. However,
when used with an adapter storing the job arguments as JSON, it would get
serialized as a simple String, resulting in deserialization also producing
a String (instead of a BigDecimal).
By using a serializer, we ensure the round trip is safe.
During upgrade deployments of applications with multiple replicas making use of
BigDecimal job arguments with a queue adapter serializing to JSON, there exists
a possible race condition, whereby a "new" replica enqueues a job with an
argument serialized using `BigDecimalSerializer`, and an "old" replica fails to
deserialize it (as it does not have `BigDecimalSerializer`).
Therefore, to ensure safe upgrades, serialization will not use
`BigDecimalSerializer` until `config.active_job.use_big_decimal_serializer` is
enabled, which can be done safely after successful deployment of Rails 7.1.
This option will be removed in Rails 7.2, when it will become the default.
This adds `:db_runtime` to `perform.active_job` notification payloads,
which is the total time taken by database queries while performing a
job. This value can be used to better understand how a job's time is
spent.
This is similar to the `:db_runtime` value added to
`process_action.action_controller` notification payloads by
`ActiveRecord::Railties::ControllerRuntime`.
Closes#35354.
Co-authored-by: Cory Gwin <gwincr11@github.com>
que 1.2 introduced a deprecation warning when specifying job options
directly within the `#enqueue` method. From 1.2 job options need to be
provided as a hash in `job_options` keyword argument.
que 1.x is not compatible with Ruby 3 yet. que 2 will be.
This commit resolves the deprecation warning and ensures future
compatibility with que 2, allowing path to Ruby 3 upgrade preserving
compatibility with que 1.0.
Co-authored-by: Adis Hasović <adis@80pct.com>
The Active Job integration tests for Que and Queue Classic assume a
connection to a PostgreSQL database running on localhost:5432 since the
psql command to create the databases does not pass the -h and -p options.
RDoc will automatically format and link API references as long as they
are not already marked up as inline code.
This commit removes markup from various API references so that those
references will link to the relevant API docs.
`config.active_record.destroy_association_async_job` should allow applications
to specify the job that will be used to destroy associated records in the
background for `has_many` associations with the `dependent: :destroy_async`
option. That's being ignored, which means the default
`ActiveRecord::DestroyAssociationAsyncJob` always destroys records in the
background. This change stops ignoring the configuration.
In other log messages like perform/performed and ActiveJob error logging, the job ID and exception message were already included, eg:
Error performing TestFailureJob (Job ID: d70ad13e-e58b-409c-a8cc-e0447fc792b5) from Resque(default) in 1446.56ms: RuntimeError (Error Message):
But log message related to retry/discard behavior from ActiveJob::Exceptions did not include the Job ID or exception message. We now include them, in a consistent format with other existing messages.
Job ID is especially useful because, while other ActiveJob-related log messages get the Job ID via tagged logging, retry-related log messages do not currently end up tagged with Job ID. And it's really useful to use Job ID to be able to collect all the log lines related to a particular job, put the error messages together with the retry and subsequent perform messages.
We also include job.executions in the enqueue_retry log message because it was available and useful, to know if this is the 1st retry or 2nd or whatever. Previously job.executions was included in retry_stopped logging, but not enqueue_retry.
Replace Que.* settings that were removed in Que 1.0 with options passed
to Que::Locker. The Locker class creates its own thread for managing
work distribution, so the Thread created in the QueJobsManager was also
removed.
Using `assert` by itself without an error message leads to test output
like the following:
```
Failure:
QueuingTest#test_current_timezone_is_kept_while_running_perform_later [.../rails/activejob/test/integration/queuing_test.rb:118]:
Expected false to be truthy.
```
These added helpers provide a more specific error message so that the
reasons for test failures become more clear:
```
Failure:
QueuingTest#test_current_timezone_is_kept_while_running_perform_later [.../rails/activejob/test/integration/queuing_test.rb:118]:
Job AJ-63fe8bf6-7062-4a56-a10a-97c37fcaa6e6 was not executed
```
Fixing the synchronous setting uncovered some more errors with Que 1.0:
exception tests with retry_on started failing due to an assertion in
Que's Active Job integration that running jobs won't nest. However, this
is not the case when running retriable jobs synchronously.
This change overrides Que's Active Job wrapper to not make this
assertion.
I also opened an issue in the que repo: https://github.com/que-rb/que/issues/329
Que.mode was removed in 6783fdd68a
and replaced by Que.run_synchronously = true per changelog:
fa4988ee33/CHANGELOG.md?plain=1#L120
It looks like it started failing because Que 1.0 was released yesterday.
I'm working on a standardized error reporting interface
(https://github.com/rails/rails/issues/43472) and it has the same need
for a `context` than Active Record's query logs.
Just like query logs, when reporting an error you want to know what the
current controller or job is, etc.
So by extracting it we can allow both API to use the same store.
Error messages from assert_enqueued_with and assert_performed_with
shows which other jobs are enqueued or performed when you get an
error. We can improve these messages to make it clearer why the job
didn't match. This one error messages now have three different formats
based on why the job didn't match
1. Didn't match because no jobs are queued at all. This now reports,
```
No performed job found with {:job=>HelloJob, :args=>[]}
No jobs where performed
```
2. Didn't match because that job class was not queued. This now reports,
```
No performed job found with {:job=>MultipleKwargsJob, :args=>[#<Person:0x00007fe38f9f8100 @id=11>]}
No jobs of class MultipleKwargsJob where performed, job classes performed: HelloJob
```
3. Doesn't match due to arguments, queue, priority or other reason. This now reports
```
No performed job found with {:job=>HelloJob, :args=>[#<Person:0x00007fe3a89fc2c8 @id=11>]}
Potential matches: {"job_class"=>"HelloJob", "job_id"=>"f8636fd9-c7a0-4565-9198-17e175f30f0e", "provider_job_id"=>nil, "queue_name"=>"default", "priority"=>nil, "arguments"=>[{"_aj_globalid"=>"gid://aj/Person/9"}], "executions"=>0, "exception_executions"=>{}, "locale"=>"en", "timezone"=>nil, "enqueued_at"=>"2021-10-27T23:19:54Z", :job=>HelloJob, :args=>[#<Person:0x00007fe3a89dce00 @id="9">], :queue=>"default", :priority=>nil}
```
Which matches the old error message, but only reports jobs that where
queued of the same class as the job you are asserting was queued.
This remove the null object as discussed in
8af78700d2.
Since it pretends to be nil but acts as thruthy in boolean context
it cause more confusion than it clean code.
`update_context` is removed in favor of `set_context` without
a block.
The ModuleSerializer does not support serializing anonymous classes
because when we try to deserialize the anonymous class, it wouldn't
know which class to use (since class name is nil).
For this reason, ModuleSerialzier now raises an error if the class
name is nil. Previously, ModuleSerializer would raise an `undefined
method `constantize' for nil:NilClass` error during deserialization.
It's not clear why the deserialization failed from the error.
In this commit, we raise an explicit error when trying to serialize
an anonymous class indicating this behaviour is not supported.
Vipul A M