Replace ORDER BY FIELD example with ORDER BY CASE
ORDER BY FIELD was MYSQL specific, which was replaced with default ORDER BY CASE in https://github.com/rails/rails/pull/45670
Every instance of `Transaction` including instances of `NullTransaction`
now respond to `invalidate!` which allows us to use transactions in
a "tell do not ask" way. This should simplify transaction invalidation
for cases were we needed check whether transaction is not a
`NullTransaction` to prevent calling `invalidate!` on `nil` state.
[ci skip]
The current documentation for #== shows an example that demonstrates an
incorrect behaviour. This change updates the example accordingly and adds a note
too.
Co-authored-by: Petrik de Heus <petrik@deheus.net>
Provides a wrapper for `new`, to provide feature parity with `create`s
ability to create multiple records from an array of hashes, using the
same notation as the `build` method on associations.
- Associations can create multiple objects from an array of hashes
- Associations can build multiple objects from an array of hashes
- Classes can create multiple objects from an array of hashes
- Classes can build multiple objects from an array of hashes (<- You are here)
Filename provides the path, whereas basename does not. It makes
sense to show the full filename, since migrations can be stored across
directories in multi-db apps.
This method was failing to return the connection to the correct state
which would break tests when eager loading the application. What's
happening is that when we check `schema_up_to_date` when the application
is eager loaded all the models are eager loaded as well so it sees that
ApplicationRecord has a connection already but that connection is set to
the wrong one. It doesn't get replaced with the right one so we see
errors about looking for tables in the wrong database. This doesn't
happen when eager loading is off because ApplicationRecord isn't set and
gets initialized with the correct connection (from it's
`connects_to/establish_connection` call).
I also refactored this to pull `needs_update` into a method and make the
loop easier to read.
While this was checking each connection for pending migrations it was
also raising as soon as it hit any pending migration. Instead we want
this to output all current pending migrations in each directory.
This change loops through all the connections to collect pending
migrations and then passes them to the error message handling if
applicable. Instead of changing the public `check_pending!`, I stopped
calling that from `check_pending_migrations` and raise the error
directory from `check_pending_migrations`. I also updated
`check_pending` to pass pending migrations from the passed connection.
I think evenutally we'll want to deprecate `check_pending!`,
it's not very useful without the other connections but since
it's public I avoided changing it.
A common misconception with transaction rollback in Rails is that
exceptions will not be re-raised from nested transactions. This is only
true for ActiveRecord::Rollback. The documentation for that class makes
that clear.
A link to that class doc has been added, as well as an explicit
restatement that all other exceptions will be re-raised.
When duplicating records, we usually want to create a new record and
don't want to keep the original lock_version. Just like timestamp
columns are not copied.
Co-authored-by: Seonggi Yang <seonggi.yang@gmail.com>
Co-authored-by: Ryohei UEDA <ueda@anipos.co.jp>
Renames `_primary_key_constraints_hash` private method to avoid implying
that it strictly represents a constraint based on the model primary key
Allows columns list used in query constraints to be configurable using
`ActiveRecord::Base.query_constraints` macro
Now when you have a has_one association and create_association is
called:
```
# begin transaction
DELETE old record
INSERT new record
# commit transaction
```
Follow-up to #46097.
Prior to #46097, `PendingMigrationsTest` created an in-memory database
to run migrations on. This prevented those migrations from affecting
other tests. #46097 switched to an on-disk database to test pending
migrations for databases other than the current
`ActiveRecord::Base.connection` database. However, #46097 used the
fixture database that other tests also use, causing those other tests to
sometimes fail. For example:
```
$ cd activerecord
$ ARCONN=sqlite3 bin/test --seed 56617 -n '/test_understands_migrations_created_out_of_order|test_migrate_using_scope_and_verbose_mode|test_migrate_using_empty_scope_and_verbose_mode/'
...
Failure:
ActiveRecord::DatabaseTasksMigrateScopeTest#test_migrate_using_empty_scope_and_verbose_mode [/rails/activerecord/test/cases/tasks/database_tasks_test.rb:1026]:
Expected "" to include "migrating".
...
Failure:
ActiveRecord::DatabaseTasksMigrateScopeTest#test_migrate_using_scope_and_verbose_mode [/rails/activerecord/test/cases/tasks/database_tasks_test.rb:990]:
Expected "No migrations ran. (using mysql scope)\n" to include "migrating".
```
This commit changes `PendingMigrationsTest` to use isolated on-disk
databases.
Closes#46374.
Co-authored-by: fatkodima <fatkodima123@gmail.com>
This changes `PostgreSQL::OID::Array#map` to invoke `subtype.map` for
each element of the array. When `subtype` is a scalar, `subtype.map`
(as implemented by `ActiveModel::Type::Value#map`) will simply call the
block with the element. When `subtype` is a non-scalar, such as a
`PostgreSQL::OID::Range`, `subtype.map` ensures that the nested scalar
values are properly mapped.
Partially reverts #45348, as an alternative fix.
Fixes#46371.
Closes#46372.
Co-authored-by: fatkodima <fatkodima123@gmail.com>
When running test, you might have pending migrations. Before Rails would
tell you to run migrations with RAILS_ENV=test.
This causes the problem that now the development environment still have
pending migrations.
The correct workflow is to run migrations in the development environment
and let Rails prepare the test database based on the schema or structure
file.
In the development environment we don't need to tell to use the
`RAILS_ENV` environment variable.
In #45924 I updated the methods on connection handler to clear all
connections regardless of role. I missed this call because it doesn't
throw deprecation warnings in the Active Record tests.
This commit adds `ActiveRecord.deprecator` and replaces all usages of
`ActiveSupport::Deprecation.warn` in `activerecord/lib` with
`ActiveRecord.deprecator`.
Additionally, this commit adds `ActiveRecord.deprecator` to
`Rails.application.deprecators` so that it can be configured using e.g.
`config.active_support.report_deprecations`.
The `rails db:version` task makes no effort today to display information for multi-database applications.
This brings up this simplistic tasks up to parity with the rest of the DB tasks that support multi-database applications.
This is accomplished by following the pattern of making the non-namespaced task (i.e. `db:version`) to output a multi-line string, iterating through all defined databases for the current environment in `database.yml`
It also adds a namespaced instance of the task for each database defined (e.g. `db:version:animals`).
**A hidden change, but critical (for the purposes that I'm working on), that is happening here** is that database configurations with `database_tasks: false` will be excluded from the logic in this task – due to the usage of the `ActiveRecord::Base.configurations.configs_for` method, which by default has `include_hidden: false`!
If a user is calling `#execute` directly with a SQL string, they can
also specify whether they'd like the query to be retried in the case
of a connection-related exception. By nature, this means that applications
looking to opt into retries across all queries can patch `#execute` to
call super with `allow_retry: true` instead of needing to reimplement the
method entirely.
The specification describes values must be URL encoded first, then any remaining single quote characters must be escaped: https://google.github.io/sqlcommenter/spec/#value-serialization
Fortunately ERB::Util.url_encode already encodes this character, unlike for example Javascript's encodeURIComponent, so this step is unneccessary.
Fix Active Record Query Logs SQLCommenter sorting
Serialized key-value pairs must be sorted by lexicographic order: https://google.github.io/sqlcommenter/spec/#sorting
Forgot to update the changelog when we changed the default values for
`config.active_record.query_log_tags_format` from `:legacy` to
`:sqlcommenter`.
This change is a followup to: https://github.com/rails/rails/issues/46179
Prior to this commit, `ciphertext_for` returned the cleartext of values
that had not yet been encrypted, such as with an unpersisted record:
```ruby
Post.encrypts :body
post = Post.create!(body: "Hello")
post.ciphertext_for(:body)
# => "{\"p\":\"abc..."
post.body = "World"
post.ciphertext_for(:body)
# => "World"
```
This commit fixes `ciphertext_for` to always return the ciphertext of
encrypted attributes:
```ruby
Post.encrypts :body
post = Post.create!(body: "Hello")
post.ciphertext_for(:body)
# => "{\"p\":\"abc..."
post.body = "World"
post.ciphertext_for(:body)
# => "{\"p\":\"xyz..."
```
Fixed a bug that caused the alias name of "group by" to be too long and the first half of the name would be the same in both cases if it was cut by max identifier length.
Fix#46285
Co-authored-by: Yusaku ONO <yono@users.noreply.github.com>
The `BeforeTypeCast` module defines `read_attribute_before_type_cast`,
`attributes_before_type_cast`, and `*_before_type_cast` attribute
methods. It also defines `attributes_for_database` and `*_for_database`
attribute methods, but no corresponding `read_attribute_for_database`
method.
This commit adds the missing `read_attribute_for_database` method.
Prior to this commit, encrypted attributes that used column default
values appeared to be encrypted on create, but were not:
```ruby
Book.encrypts :name
book = Book.create!
book.name
# => "<untitled>"
book.name_before_type_cast
# => "{\"p\":\"abc..."
book.reload.name_before_type_cast
# => "<untitled>"
```
This commit ensures attributes with column default values are encrypted:
```ruby
Book.encrypts :name
book = Book.create!
book.name
# => "<untitled>"
book.name_before_type_cast
# => "{\"p\":\"abc..."
book.reload.name_before_type_cast
# => "{\"p\":\"abc..."
```
The existing "support encrypted attributes defined on columns with
default values" test in `encryptable_record_test.rb` shows the intended
behavior, but it was not failing without a `reload`.
In a multi-db world, delegating from `Base` to the handler doesn't make
much sense. Applications should know when they are dealing with a single
connection (Base.connection) or the handler which deals with multiple
connections. Delegating to the connection handler from Base breaks this
contract and makes behavior confusing. I think eventually a new object
will replace the handler altogether but for now I'd like to just
separate concerns to avoid confusion.
We don't need to pass the db configs to `check_pending_migrations`
because we always want to loop through all of them so the argument
doesn't gain us anything. It's also a private API so applications
shouldn't be consuming this method (and therefore makes it even less
necessary to allow configs to be passed in)
Follow-up to #44625.
In #44625, the `SerializeCastValue` module was added to allow types to
avoid a redundant call to `cast` when serializing a value for the
database. Because it introduced a new method (`serialize_cast_value`)
that was not part of the `ActiveModel::Type::Value` contract, it was
designed to be opt-in. Furthermore, to guard against incompatible
`serialize` and `serialize_cast_value` implementations in types that
override `serialize` but (unintentionally) inherit `serialize_cast_value`,
types were required to explicitly include the `SerializeCastValue`
module to activate the optimization. i.e. It was not sufficient just to
have `SerializeCastValue` in the ancestor chain.
The `SerializeCastValue` module is not part of the public API, and there
are no plans to change that, which meant user-created custom types could
not benefit from this optimization.
This commit changes the opt-in condition such that it is sufficient for
the owner of the `serialize_cast_value` method to be the same or below
the owner of the `serialize` method in the ancestor chain. This means
a user-created type that only overrides `cast`, **not** `serialize`,
will now benefit from the optimization. For example, a type like:
```ruby
class DowncasedString < ActiveModel::Type::String
def cast(value)
super&.downcase
end
end
```
As demonstrated in the benchmark below, this commit does not change the
current performance of the built-in Active Model types. However, for a
simple custom type like `DowncasedString`, the performance of
`value_for_database` is twice as fast. For types with more expensive
`cast` operations, the improvement may be greater.
**Benchmark**
```ruby
# frozen_string_literal: true
require "benchmark/ips"
require "active_model"
class DowncasedString < ActiveModel::Type::String
def cast(value)
super&.downcase
end
end
ActiveModel::Type.register(:downcased_string, DowncasedString)
VALUES = {
my_big_integer: "123456",
my_boolean: "true",
my_date: "1999-12-31",
my_datetime: "1999-12-31 12:34:56 UTC",
my_decimal: "123.456",
my_float: "123.456",
my_immutable_string: "abcdef",
my_integer: "123456",
my_string: "abcdef",
my_time: "1999-12-31T12:34:56.789-10:00",
my_downcased_string: "AbcDef",
}
TYPES = VALUES.to_h { |name, value| [name, name.to_s.delete_prefix("my_").to_sym] }
class MyModel
include ActiveModel::API
include ActiveModel::Attributes
TYPES.each do |name, type|
attribute name, type
end
end
attribute_set = MyModel.new(VALUES).instance_variable_get(:@attributes)
TYPES.each do |name, type|
attribute = attribute_set[name.to_s]
Benchmark.ips do |x|
x.report(type.to_s) { attribute.value_for_database }
end
end
```
**Before**
```
big_integer 2.986M (± 1.2%) i/s - 15.161M in 5.078972s
boolean 2.980M (± 1.1%) i/s - 15.074M in 5.059456s
date 2.960M (± 1.1%) i/s - 14.831M in 5.011355s
datetime 1.368M (± 0.9%) i/s - 6.964M in 5.092074s
decimal 2.930M (± 1.2%) i/s - 14.911M in 5.089048s
float 2.932M (± 1.3%) i/s - 14.713M in 5.018512s
immutable_string 3.013M (± 1.3%) i/s - 15.239M in 5.058085s
integer 1.603M (± 0.8%) i/s - 8.096M in 5.052046s
string 2.977M (± 1.1%) i/s - 15.168M in 5.094874s
time 1.338M (± 0.9%) i/s - 6.699M in 5.006046s
downcased_string 1.394M (± 0.9%) i/s - 7.034M in 5.046972s
```
**After**
```
big_integer 3.016M (± 1.0%) i/s - 15.238M in 5.053005s
boolean 2.965M (± 1.3%) i/s - 15.037M in 5.071921s
date 2.924M (± 1.0%) i/s - 14.754M in 5.046294s
datetime 1.435M (± 0.9%) i/s - 7.295M in 5.082498s
decimal 2.950M (± 0.9%) i/s - 14.800M in 5.017225s
float 2.964M (± 0.9%) i/s - 14.987M in 5.056405s
immutable_string 2.907M (± 1.4%) i/s - 14.677M in 5.049194s
integer 1.638M (± 0.9%) i/s - 8.227M in 5.022401s
string 2.971M (± 1.0%) i/s - 14.891M in 5.011709s
time 1.454M (± 0.9%) i/s - 7.384M in 5.079993s
downcased_string 2.939M (± 0.9%) i/s - 14.872M in 5.061100s
```
Previously, most version of the Migration class would call `super` in
compatible_table_definition to ensure that they append all of the later
Migration verion's TableDefinition modules. However, the latest
Migration class did not do this and prepended its own TableDefinition
because there was no super method for it to call.
This led to an issue where Action Text migrations started failing on
Rails main, after migration option validation was added in e6da3eb. The
new functionality was not supposed to affect V6_0 Migrations, but it was
because both V6_1 and V7_0 were not calling super.
This commit fixes the issue by correctly calling super in both classes.
It also adds a compatible_table_definition method to Current that
returns the value passed in so that all of the versioned Migrations can
always return super.
In #38235 I moved advisory locks to their own named connections, then
in #39758 the advisory lock was left on Base.connection but then moved
it it's own connection handler. I believe with #45450 that this change
was made obsolete and can be returned to the prior behavior without
having to open an additional connection. The tests added pass and I also
tested this in my local demo to ensure that this is working correctly.
When I originally changed the behavior here Matthew noted that this
could be surprising for some setups that expect only one connection for
a running migration. I thought there was an issue related to this but I
can't find it.
This commit adds a step to validate the options that are used when managing columns and tables in migrations.
The intention is to only validate options for new migrations that are added. Invalid options used in old migrations are silently ignored like they always have been.
Fixes#33284Fixes#39230
Co-authored-by: George Wambold <georgewambold@gmail.com>
We recently let a few very easy to avoid warnings get merged.
The root cause is that locally the test suite doesn't run in
verbose mode unless you explictly pass `-w`.
On CI warnings are enabled, but there is no reason to look at the
build output unless something is failing. And even if one wanted
to do that, that would be particularly work intensive since warnings
may be specific to a Ruby version etc.
Because of this I believe we should:
- Always run the test suite with warnings enabled.
- Raise an error if a warning is unexpected.
We've been using this pattern for a long time at Shopify both in private
and public repositories.
> Sets readonly attributes for the returned relation.
The current documentation gives the impression that it can manipulate
`attr_readonly`.
Co-authored-by: Petrik de Heus <petrik@deheus.net>
When `column` is passed as an array and the column does not exist,
then `index_exists?` check would fail to return early which would cause
`index_name_for_remove` to raise an error instead of ignoring the
missing column. This change ensures that `defined_for` checks the
column options in addition to the column argument.
Fixes#46196
This fixes the following Rubocop offenses:
```
activerecord/lib/active_record/query_logs.rb:96:9: C: [Correctable] Layout/CaseIndentation: Indent when as deep as case.
when :legacy
^^^^
activerecord/lib/active_record/query_logs.rb:98:9: C: [Correctable] Layout/CaseIndentation: Indent when as deep as case.
when :sqlcommenter
^^^^
```
In Active Record `connection_class` already means "the class in your
application that established the connection" so I would prefer it only
have that one meaning. This change swaps `connection_class` for
`adapter_class` where applicable.
This code block was removed in a previous PR, but it had
a side effect of causing certian timestamp fields to be
changed from `datetime(6)` to `datetime` when performing a schema
migration in the 6.1 version of the migration class. This is due
to `options[:precision]` being set to `nil` and therefore
not being set to the default of 6 later on in the code path.
Instead of hosting the connection logic in the command object, the
database adapter should be responsible for connecting to a console session.
This patch moves #find_cmd_and_exec to the adapter and exposes a new API to
lookup the adapter class without instantiating it.
Co-authored-by: Paarth Madan <paarth.madan@shopify.com>
Most model attribute types try to cast a given value before serializing
it. This allows uncast values to be passed to finder methods and still
be serialized appropriately. However, when persisting a model, this
cast is unnecessary because the value will already have been cast by
`ActiveModel::Attribute#value`.
To eliminate the overhead of a 2nd cast, this commit introduces a
`ActiveModel::Type::SerializeCastValue` module. Types can include this
module, and their `serialize_cast_value` method will be called instead
of `serialize` when serializing an already-cast value.
To preserve existing behavior of any user types that subclass Rails'
types, `serialize_after_cast` will only be called if the type itself
(not a superclass) includes `ActiveModel::Type::SerializeCastValue`.
This also applies to type decorators implemented via `DelegateClass`.
Benchmark script:
```ruby
require "active_model"
require "benchmark/ips"
class ActiveModel::Attribute
alias baseline_value_for_database value_for_database
end
VALUES = {
my_big_integer: "123456",
my_boolean: "true",
my_date: "1999-12-31",
my_datetime: "1999-12-31 12:34:56 UTC",
my_decimal: "123.456",
my_float: "123.456",
my_immutable_string: "abcdef",
my_integer: "123456",
my_string: "abcdef",
my_time: "1999-12-31T12:34:56.789-10:00",
}
TYPES = VALUES.to_h { |name, value| [name, name.to_s.delete_prefix("my_").to_sym] }
class MyModel
include ActiveModel::API
include ActiveModel::Attributes
TYPES.each do |name, type|
attribute name, type
end
end
TYPES.each do |name, type|
$attribute_set ||= MyModel.new(VALUES).instance_variable_get(:@attributes)
attribute = $attribute_set[name.to_s]
puts "=" * 72
Benchmark.ips do |x|
x.report("#{type} before") { attribute.baseline_value_for_database }
x.report("#{type} after") { attribute.value_for_database }
x.compare!
end
end
```
Benchmark results:
```
========================================================================
Warming up --------------------------------------
big_integer before 100.417k i/100ms
big_integer after 260.375k i/100ms
Calculating -------------------------------------
big_integer before 1.005M (± 1.0%) i/s - 5.121M in 5.096498s
big_integer after 2.630M (± 1.0%) i/s - 13.279M in 5.050387s
Comparison:
big_integer after: 2629583.6 i/s
big_integer before: 1004961.2 i/s - 2.62x (± 0.00) slower
========================================================================
Warming up --------------------------------------
boolean before 230.663k i/100ms
boolean after 299.262k i/100ms
Calculating -------------------------------------
boolean before 2.313M (± 0.7%) i/s - 11.764M in 5.085925s
boolean after 3.037M (± 0.6%) i/s - 15.262M in 5.026280s
Comparison:
boolean after: 3036640.8 i/s
boolean before: 2313127.8 i/s - 1.31x (± 0.00) slower
========================================================================
Warming up --------------------------------------
date before 148.821k i/100ms
date after 298.939k i/100ms
Calculating -------------------------------------
date before 1.486M (± 0.6%) i/s - 7.441M in 5.006091s
date after 2.963M (± 0.8%) i/s - 14.947M in 5.045651s
Comparison:
date after: 2962535.3 i/s
date before: 1486459.4 i/s - 1.99x (± 0.00) slower
========================================================================
Warming up --------------------------------------
datetime before 92.818k i/100ms
datetime after 136.710k i/100ms
Calculating -------------------------------------
datetime before 920.236k (± 0.6%) i/s - 4.641M in 5.043355s
datetime after 1.366M (± 0.8%) i/s - 6.836M in 5.003307s
Comparison:
datetime after: 1366294.1 i/s
datetime before: 920236.1 i/s - 1.48x (± 0.00) slower
========================================================================
Warming up --------------------------------------
decimal before 50.194k i/100ms
decimal after 298.674k i/100ms
Calculating -------------------------------------
decimal before 494.141k (± 1.4%) i/s - 2.510M in 5.079995s
decimal after 3.015M (± 1.0%) i/s - 15.232M in 5.052929s
Comparison:
decimal after: 3014901.3 i/s
decimal before: 494141.2 i/s - 6.10x (± 0.00) slower
========================================================================
Warming up --------------------------------------
float before 217.547k i/100ms
float after 298.106k i/100ms
Calculating -------------------------------------
float before 2.157M (± 0.8%) i/s - 10.877M in 5.043292s
float after 2.991M (± 0.6%) i/s - 15.203M in 5.082806s
Comparison:
float after: 2991262.8 i/s
float before: 2156940.2 i/s - 1.39x (± 0.00) slower
========================================================================
Warming up --------------------------------------
immutable_string before
163.287k i/100ms
immutable_string after
298.245k i/100ms
Calculating -------------------------------------
immutable_string before
1.652M (± 0.7%) i/s - 8.328M in 5.040855s
immutable_string after
3.022M (± 0.9%) i/s - 15.210M in 5.033151s
Comparison:
immutable_string after: 3022313.3 i/s
immutable_string before: 1652121.7 i/s - 1.83x (± 0.00) slower
========================================================================
Warming up --------------------------------------
integer before 115.383k i/100ms
integer after 159.702k i/100ms
Calculating -------------------------------------
integer before 1.132M (± 0.8%) i/s - 5.769M in 5.095041s
integer after 1.641M (± 0.5%) i/s - 8.305M in 5.061893s
Comparison:
integer after: 1640635.8 i/s
integer before: 1132381.5 i/s - 1.45x (± 0.00) slower
========================================================================
Warming up --------------------------------------
string before 163.061k i/100ms
string after 299.885k i/100ms
Calculating -------------------------------------
string before 1.659M (± 0.7%) i/s - 8.316M in 5.012609s
string after 2.999M (± 0.6%) i/s - 15.294M in 5.100008s
Comparison:
string after: 2998956.0 i/s
string before: 1659115.6 i/s - 1.81x (± 0.00) slower
========================================================================
Warming up --------------------------------------
time before 98.250k i/100ms
time after 133.463k i/100ms
Calculating -------------------------------------
time before 987.771k (± 0.7%) i/s - 5.011M in 5.073023s
time after 1.330M (± 0.5%) i/s - 6.673M in 5.016573s
Comparison:
time after: 1330253.9 i/s
time before: 987771.0 i/s - 1.35x (± 0.00) slower
```
Commit 37d1429ab1 introduced the DummyERB to avoid loading the environment when
running `rake -T`.
The DummyCompiler simply replaced all output from `<%=` with a fixed string and
removed everything else. This worked okay when it was used for YAML values.
When using `<%=` within a YAML key, it caused an error in the YAML parser,
making it impossible to use ERB as you would expect. For example a
`database.yml` file containing the following should be possible:
development:
<% 5.times do |i| %>
shard_<%= i %>:
database: db/development_shard_<%= i %>.sqlite3
adapter: sqlite3
<% end %>
Instead of using a broken ERB compiler we can temporarily use a
`Rails.application.config` that does not raise an error when configurations are
accessed which have not been set as described in #35468.
This change removes the `DummyCompiler` and uses the standard `ERB::Compiler`.
It introduces the `DummyConfig` which delegates all known configurations to the
real `Rails::Application::Configuration` instance and returns a dummy string for
everything else. This restores the full ERB capabilities without compromising on
speed when generating the rake tasks for multiple databases.
Deprecates `config.active_record.suppress_multiple_database_warning`.
Prior to this change
t.virtual :column_name, type: :datetime
would erroneously produce the same result as
t.virtual :column_name, type: :datetime, precision: nil
This is because the code path for `virtual` skipped the default lookup:
- `t.datetime` is delegated to the `column` method
- `column` sees `type == :datetime` and sets a default `precision` is none was given
- `column` calls `new_column_definition` to add the result to `@columns_hash`
- `t.virtual` is delegated to the `column` method
- `column` sees `type == :virtual`, not `:datetime`, so skips the default lookup
- `column` calls `new_column_definition` to add the result to `@columns_hash`
- `new_column_definition` sees `type == :virtual`, so sets `type = options[:type]`
By moving the default lookup, we get consistent code paths:
- `t.datetime` is delegated to the `column` method
- `column` calls `new_column_definition` to add the result to `@columns_hash`
- `new_column_definition` sees `type == :datetime` and sets a default `precision` is none was given
- `t.virtual` is delegated to the `column` method
- `column` calls `new_column_definition` to add the result to `@columns_hash`
- `new_column_definition` sees `type == :virtual`, so sets `type = options[:type]`
- `new_column_definition` sees `type == :datetime` and sets a default `precision` is none was given
Active Record erroneously uses a precision of `nil` instead of the default
precision, when adding a virtual datetime column.
This tests that behavior ahead of fixing it.
Problem:
Though, the `MessageVerifier` that powers `signed_id` supports both
`expires_in` and `expires_at`, `signed_id` only supports `expires_in`.
Because of this, generating signed_id that expires at a certain time is
somewhat tedious. Imagine issuing a coupon that is valid only for a
day.
Solution:
Add `expires_at` option to `signed_id` to generate signed ids that
expire at the given time.
Building on the work done in #44576 and #44591, we extend the logic that automatically
reconnects broken db connections to take into account a timeout limit. This ensures
that retries + reconnects are slow-query aware, and that we don't retry queries
if a given amount of time has already passed since the query was first tried.
This value will default to 5 seconds, but can be adjusted via the `connection_retry_timeout`
config.
Even if tests are "skipped" by `Minitest::Assertions#skip`
`ensure` is executed because tests methods are Ruby methods.
Adding conditions outside of the test methods, entire test methods are
executed only when they are necessary.
Before f56b4187be this constant was being
loaded in the railtie, but it was removed since we didn't needed it
anymore.
It causes some application to break if they try to reference the
subscriber before `ActiveRecord::Base` is loaded.
This PR calls `ActiveRecord::Base` directly on `establish_connection`
and `connection` rather than delegating. While this doesn't have much
effect right now, I'm working on moving the database tasks away from
their reliance on Base and eventually we'll need to pass a class through
to these adapter tasks. This change prepares these adapter tasks for
that change.
This avoids problems when complex data structures are mutated _after_
being handed to ActiveRecord for processing. For example false hits in
the query cache.
Fixes#46044
In f36c261a44 support for the ssl-mode option was added for MySQL when
using the dbconsole command and MySQLDatabaseTasks.
However, the mysql2 gem uses `ssl_mode` instead of `sslmode`:
ba4d46551d/lib/mysql2/client.rb (L51)
As `ssl_mode` is what should be used in the database.yml configuration
for MySQL we should use it as well for the dbconsole command and
MySQLDatabaseTasks.
Co-authored-by: Chris Gunther <chris@room118solutions.com>
When I moved `SchemaMigration` and `InternalMetadata` away from inheriting
from `ActiveRecord::Base` I accidentally set the logger tags to `self`
rather than `self.class` which meant they were accidentally logging
the instance rather than the class name. While fixing these I realized I
missed a few tags for other queries and have fixed those as well.
According to the MySQL documentation, database connections default to
ssl-mode=PREFERRED. But PREFERRED doesn't verify the server's identity:
The default setting, --ssl-mode=PREFERRED, produces an encrypted
connection if the other default settings are unchanged. However, to
help prevent sophisticated man-in-the-middle attacks, it is
important for the client to verify the server’s identity. The
settings --ssl-mode=VERIFY_CA and --ssl-mode=VERIFY_IDENTITY are a
better choice than the default setting to help prevent this type of
attack. VERIFY_CA makes the client check that the server’s
certificate is valid. VERIFY_IDENTITY makes the client check that
the server’s certificate is valid, and also makes the client check
that the host name the client is using matches the identity in the
server’s certificate.
https://dev.mysql.com/doc/refman/8.0/en/using-encrypted-connections.html
However both the Rails::DBConsole command and the MySQLDatabaseTasks
ignore the ssl-mode option, making the connection fallback to PREFERRED.
Adding ssl-mode to the forwarded options makes sure the expected mode is
passed to the connection.
In https://github.com/rails/rails/pull/45796 I overlooked that
`ActiveRecord::LogSubscriber#sql` wasn't only logging the SQL query
but was also responsible for collecting the `:db_runtime` metric.
Ultimately I think it is cleaner to move this concern to `RuntimeRegistry`.
Since changing internal metadata to no longer inherit from Base
in #45982 I accidentally changed the behavior when a key's value is the
same. Prior to this change the record would not be updated if the
environment key was found an the value had not changed. So instead of
just checking whether we have an entry here we also need to check if the
value should actually be updated, otherwise we should return the entry.
This clean up wraps the messages at 80 characters and clarifies the
changes needed. 99% of apps won't see this warning as they aren't
accessing MigrationContext directly. This is only for libraries or apps
that are passing a custom `SchemaMigration` or `InternalMetadata` class
so they get a more helpful error.
In addition I moved the `NullSchemaMigration` up to the top of the class
to match InternalMetadata.
Following #45924 sometimes these calls will throw deprecation warnings
depending on the test order and whether there are connections left
behind from other tests. I attempted to prevent leaking connections but
when using fixtures it's pretty hard to avoid because the
`teardown_fixture_connections` will often put connections back in place.
It is eaiser to ensure these calls opt into the new behavior to avoid
deprecation warnings.
Followup to #45908 to match the same behavior as SchemaMigration
Previously, InternalMetadata inherited from ActiveRecord::Base. This is
problematic for multiple databases and resulted in building the code in
AbstractAdapter that was previously there. Rather than hacking around
the fact that InternalMetadata inherits from Base, this PR makes
InternalMetadata an independent object. Then each connection can get it's
own InternalMetadata object. This change required defining the methods
that InternalMetadata was depending on ActiveRecord::Base for (ex
create!). I reimplemented only the methods called by the framework as
this class is no-doc's so it doesn't need to implement anything beyond
that. Now each connection gets it's own InternalMetadata object which
stores the connection.
This change also required adding a NullInternalMetadata class for cases
when we don't have a connection yet but still need to copy migrations
from the MigrationContext. Ultimately I think this is a little weird -
we need to do so much work to pick up a set of files? Maybe something to
explore in the future.
Aside from removing the hack we added back in #36439 this change will
enable my work to stop clobbering and depending directly on
Base.connection in the rake tasks. While working on this I discovered
that we always have a ActiveRecord::InternalMetadata because the
connection is always on Base in the rake tasks. This will free us up
to do less hacky stuff in the migrations and tasks.
Both schema migration and internal metadata are blockers to removing
`Base.connection` and `Base.establish_connection` from rake tasks, work
that is required to drop the reliance on `Base.connection` which will
enable more robust (and correct) sharding behavior in Rails..
Previously, only bold text was supported via a positional argument.
This allows for bold, italic, and underline options to be specified
for colored logs.
```ruby
info color("Hello world!", :red, bold: true, underline: true)
```
The issue fixed by the commit that introduced that entry only existed
in the main branch, so it isn't really a released change worthy of a
CHANGELOG entry.
Previously, SchemaMigration inherited from ActiveRecord::Base. This is
problematic for multiple databases and resulted in building the code in
AbstractAdapter that was previously there. Rather than hacking around
the fact that SchemaMigration inherits from Base, this PR makes
SchemaMigration an independent object. Then each connection can get it's
own SchemaMigration object. This change required defining the methods
that SchemaMigration was depending on ActiveRecord::Base for (ex
create!). I reimplemented only the methods called by the framework as
this class is no-doc's so it doesn't need to implement anything beyond
that. Now each connection gets it's own SchemaMigration object which
stores the connection. I also decided to update the method names (create
-> create_version, delete_by -> delete_version, delete_all ->
delete_all_versions) to be more explicit.
This change also required adding a NullSchemaMigraiton class for cases
when we don't have a connection yet but still need to copy migrations
from the MigrationContext. Ultimately I think this is a little weird -
we need to do so much work to pick up a set of files? Maybe something to
explore in the future.
Aside from removing the hack we added back in #36439 this change will
enable my work to stop clobbering and depending directly on
Base.connection in the rake tasks. While working on this I discovered
that we always have a `ActiveRecord::SchemaMigration` because the
connection is always on `Base` in the rake tasks. This will free us up
to do less hacky stuff in the migrations and tasks.
This is a partial fix for #45906 to allocate less objects when iterating
through pool configs. My original attempt to fix this wasn't feasible
due to how the query cache works. The idea to use a dedicated `each_*`
came from jonathanhefner and I incorporated it here.
Using the following script:
```
Benchmark.memory do |x|
x.report("all") do
ActiveRecord::Base.connection_handler.all_connection_pools.each { }
end
x.report("each") do
ActiveRecord::Base.connection_handler.each_connection_pool { }
end
x.compare!
end
```
We can see that less objects are allocated with the each method. This is
even more important now that we are calling `all_connection_pools` in
more placed since #45924 and #45961.
```
Calculating -------------------------------------
all 408.000 memsize ( 0.000 retained)
7.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
each 328.000 memsize ( 0.000 retained)
3.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
Comparison:
each: 328 allocated
all: 408 allocated - 1.24x more
```
Note that if this is run again on this branch `all_connection_pools`
will have even more allocations due to the deprecation warning that was
introduced. However removing that shows similar numbers to the benchmark
here.
Co-authored-by: Jonathan Hefner <jonathan@hefner.pro>
This PR enables `Minitest/AssertRaisesWithRegexpArgument` cop
and it suppresses the new warning below.
```console
% bundle exec rubocop
(snip)
Offenses:
activerecord/test/cases/adapters/sqlite3/sqlite3_adapter_test.rb:111:9: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises(ActiveRecord::StatementInvalid, /TypeError/) do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
activerecord/test/cases/adapters/sqlite3/sqlite3_adapter_test.rb:628:9: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises(ActiveRecord::StatementInvalid, /SQLite3::ReadOnlyException/) do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
railties/test/application/rake/multi_dbs_test.rb:307:13: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises RuntimeError, /You're using a multiple database application/ do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
railties/test/application/rake/multi_dbs_test.rb:311:13: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises RuntimeError, /You're using a multiple database application/ do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
railties/test/application/rake/multi_dbs_test.rb:336:13: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises RuntimeError, /You're using a multiple database application/ do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
railties/test/application/rake/multi_dbs_test.rb:361:13: C: Minitest/AssertRaisesWithRegexpArgument:
Do not pass regular expression literals to assert_raises. Test the resulting exception.
assert_raises RuntimeError, /You're using a multiple database application/ do
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
```
The last argument of `assert_raises` is a custom message string to help explain failures.
So, it's not the argument that `exception.message` is compared to.
`assert_raises` returns a raised exception and can be used to match against a regular expression.
And it updates the dependency version of rubocop-minitest in the Gemfile.
Because `Minitest/AssertRaisesWithRegexpArgument` cop was introduced in minitest 0.22.
https://github.com/rubocop/rubocop-minitest/releases/tag/v0.22.0
Following on #45924 I realized that `all_connection_pools` and
`connection_pool_list` don't make much sense as separate methods and
should follow the same deprecation as the other methods on the handler
here. So this PR deprecates `all_connection_pools` in favor of
`connection_pool_list` with an explicit argument of the role or `:all`.
Passing `nil` will throw a deprecation warning to get applications to
be explicit about behavior they expect.
Previously when I implemented multiple database roles in Rails there
were two handlers so it made sense for the methods
`active_connections?`, `clear_active_connections!`,
`clear_reloadable_connections!`, `clear_all_connections!`, and
`flush_idle_connections!` to only operate on the current (or passed)
role and not all pools regardless of role. When I removed this and moved
all the pools to the handler maintained by a pool manager, I left these
methods as-is to preserve the original behavior.
This made sense because I thought these methods were only called by
applications and not called by Rails. I realized yesterday that some of
these methods (`flush_idle_connections!`, `clear_active_connections!`,
and `clear_reloadable_connections!` are all called on boot by the
Active Record railtie.
Unfortunately this means that applications using multiple databases
aren't getting connections flushed or cleared on boot for any connection
but the writing ones.
The change here continues existing behavior if a role like reading is
passed in directly. Otherwise if the role is `nil` (which is the new
default` we fall back to all connections and issue a deprecation
warning. This will be the new default behavior in the future. In order
to easily allow turning off the deprecation warning I've added an `:all`
argument that will use all pools but no warning. The deprecation warning
will only fire if there is more than one role in the pool manager,
otherwise we assume prior behavior.
This bug would have only affected applications with more than one role
and only when these methods are called outside the context of a
`connected_to` block. These methods no longer consider the set
`current_role` and applications need to be explicit if they don't want
these methods to operate on all pools.
This commit makes two performance tweaks to the dispatch_cache in
Arel::Visitors::Visitor.
First, it interns the method name into a symbol rather than a string.
send expects a symbol, so this avoids Ruby needing to intern that string
itself.
Second, it changes the dispatch cache to an identity hash. Classes are
already compared by identity, but Ruby is able to perform the lookup
more quickly if it is explicit on the hash.
Add ability to use hash with columns and aliases inside #select method.
Post
.joins(:comments)
.select(
posts: { id: :post_id, title: :post_title },
comments: { id: :comment_id, body: :comment_body}
)
instead
Post
.joins(:comments)
.select(
"posts.id as post_id, posts.title as post_title,
comments.id as comment_id, comments.body as comment_body"
)
Co-authored-by: Josef Šimánek <193936+simi@users.noreply.github.com>
Co-authored-by: Jean byroot Boussier <19192189+casperisfine@users.noreply.github.com>
During where clause generation we search for scope type for the model.
We can do this with Module#!= instead as long as we grab the final scope type after the loop.
This is a small, but significant performance improvement.
Co-authored-by: John Hawthorn <jhawthorn@github.com>
Updating the documentation for setting `self.ignored_columns` on an ActiveRecord model to append to the existing value instead of overwriting the existing array.
Appending to the existing value, `+=`, is helpful for the scenario when 2 developers set `ignored_columns` in different parts of the file and the second one overwrites the first one.
When source and target classes have a different set of attributes adapts
attributes such that the extra attributes from target are added.
Fixes#41195
Co-authored-by: SampsonCrowley <sampsonsprojects@gmail.com>
Co-authored-by: Jonathan Hefner <jonathan@hefner.pro>
[`composed_of`](https://api.rubyonrails.org/classes/ActiveRecord/Aggregations/ClassMethods.html#method-i-composed_of)
is a feature that is not widely used and its API is somewhat confusing,
especially for beginners. It was even deprecated for a while, but 10
years after its deprecation, it's still here.
The Rails documentation includes these examples including mapping:
```ruby
composed_of :temperature, mapping: %w(reading celsius)
composed_of :balance, class_name: "Money", mapping: %w(balance amount)
composed_of :address, mapping: [ %w(address_street street),
%w(address_city city) ]
```
Hashes are accepted kind-of accidentally for the `mapping` option. Using
a hash, instead of an array or array of arrays makes the documentation
more beginner-friendly.
```ruby
composed_of :temperature, mapping: { reading: :celsius }
composed_of :balance, class_name: "Money", mapping: { balance: :amount }
composed_of :address, mapping: { address_street: :street, address_city:
:city }
```
Before Ruby 1.9, looping through a hash didn't have deterministic order,
and the mapping order is important, as that's the same order Rails uses
when initializing the value object. Since Ruby 1.9, this isn't an issue
anymore, so we can change the documentation to use hashes instead.
This commit changes the documentation for `composed_of`, clarifying that
any key-value format (both a `Hash` and an `Array`) are accepted for the
`mapping` option. It also adds tests to ensure hashes are also accepted.
While working on another PR I noticed that we have 6 different tasks for
loading the schema; 3 for all dbs, 3 for namespaced dbs. They are:
* `db:schema:load`
* `db:schema:load:namespace`
* `db:test:load_schema`
* `db:test:load_schema:namespace`
* `db:test:load` (calls `db:test:load_schema` only)
* `db:test:load:namespace` (calls `db:test:load_schema:namespace` only)
I've removed the last two because they didn't call anything except for
another task and they were already private. I believe the original
`db:test:load` and `db:test:load:namescace` were added to handle whether
to load sql or ruby schema fules, but we no longer need that (it's
handled by the `load_schema` method directly now).
In addition I removed `:load_config, :check_protected_environments` from
the `db:schema:load:namespace` task because `purge` already calls these
so we don't need to do it twice.
I tried to remove `db:test:load_schema:namespace` and call
`db:schema:load:namespace` instead but we can't do that because
`db:test:load_schema:namespace` needs to set the environment to "test",
not use the current environment. `db:schema:load` also can't be used
instead of `db:test:load_schema` because it tries to load both dev and
test schemas, but it's callers want just the test schema.
This PR is similar to #36439 which moved schema migrations to the
connection and migration context. This PR does the same thing for the
internal metadata class.
Previously the metadata class was only accessible on Base which makes it
a little weird when switching connections. This didn't really cause any
issues but I noticed it when working on a PR to change how connections
work in the database rake tasks.
In a multiple database environment it makes sense to access the
`ar_internal_metadata` on the existing connection rather than on
`Base.connection`. I previously didn't implement this because I felt we
wanted only one place to ask for the metadata information but recently
realized that we were still recording metadata on all the databases, so
we might as well record it correctly.
Applications should notice no change in behavior because this is only
accessed through the rake tasks. Additionally behavior between schema
migrations and internal metadata are now more similar. Eventually I'd
like neither of these to inherit from or rely on Base, but we have a
lot more work to do before that's possible.
This adds support for reading binary column default values before the column data is read from the database.
This makes binary columns behave more like other column types with default values
In Ruby trunk date changed to use the gregorian calendar in
https://github.com/ruby/date/pull/73.
This had the result that dates not in the gregorian calendar can't be
compared to times not in the gregorian calendar. Since this test isn't
testing datetime comparison but rather that bc timestamps can be stored
in the database, we can compare with `Time.new(0) - 1.week` rather than
`Date.new(0) - 1.week`. I think this might be a bug in Ruby but I'm not
sure there's a great fix. For now however, let's get this test passing.
sampatbadhe