This commit speeds up rendering partials by caching the variable name
calculation on the template. The variable name is based on the "virtual
path" used for looking up the template. The same virtual path
information lives on the template, so we can just ask the cached
template object for the variable.
This benchmark takes a couple files, so I'll cat them below:
```
[aaron@TC ~/g/r/actionview (speed-up-partials)]$ cat render_benchmark.rb
require "benchmark/ips"
require "action_view"
require "action_pack"
require "action_controller"
class TestController < ActionController::Base
end
TestController.view_paths = [File.expand_path("test/benchmarks")]
controller_view = TestController.new.view_context
result = Benchmark.ips do |x|
x.report("render") do
controller_view.render("many_partials")
end
end
[aaron@TC ~/g/r/actionview (speed-up-partials)]$ cat test/benchmarks/test/_many_partials.html.erb
Looping:
<ul>
<% 100.times do |i| %>
<%= render partial: "list_item", locals: { i: i } %>
<% end %>
</ul>
[aaron@TC ~/g/r/actionview (speed-up-partials)]$ cat test/benchmarks/test/_list_item.html.erb
<li>Number: <%= i %></li>
```
Benchmark results (master):
```
[aaron@TC ~/g/r/actionview (master)]$ be ruby render_benchmark.rb
Warming up --------------------------------------
render 41.000 i/100ms
Calculating -------------------------------------
render 424.269 (± 3.5%) i/s - 2.132k in 5.031455s
```
Benchmark results (this branch):
```
[aaron@TC ~/g/r/actionview (speed-up-partials)]$ be ruby render_benchmark.rb
Warming up --------------------------------------
render 50.000 i/100ms
Calculating -------------------------------------
render 521.862 (± 3.8%) i/s - 2.650k in 5.085885s
```
Active Record uses `scoping` to delegate to named scopes from relations
for propagating the chaining source scope. It was needed to restore the
source scope in named scopes, but it was caused undesired behavior that
pollute all class level querying methods.
Example:
```ruby
class Topic < ActiveRecord::Base
scope :toplevel, -> { where(parent_id: nil) }
scope :children, -> { where.not(parent_id: nil) }
scope :has_children, -> { where(id: Topic.children.select(:parent_id)) }
end
# Works as expected.
Topic.toplevel.where(id: Topic.children.select(:parent_id))
# Doesn't work due to leaking `toplevel` to `Topic.children`.
Topic.toplevel.has_children
```
Since #29301, the receiver in named scopes has changed from the model
class to the chaining source scope, so the polluting class level
querying methods is no longer required for that purpose.
Fixes#14003.
With this benchmark:
require "bundler/setup"
require "active_record"
require "benchmark/ips"
# This connection will do for database-independent bug reports.
ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")
ActiveRecord::Schema.define do
create_table :posts, force: true do |t|
end
end
class Post < ActiveRecord::Base
end
new_post = Post.new
Benchmark.ips do |b|
b.report("present?") do
new_post.present?
end
b.report("blank?") do
new_post.blank?
end
end
Before:
Warming up --------------------------------------
present? 52.147k i/100ms
blank? 53.077k i/100ms
Calculating -------------------------------------
present? 580.184k (±21.8%) i/s - 2.555M in 5.427085s
blank? 601.537k (± 9.2%) i/s - 2.972M in 5.003503s
After:
Warming up --------------------------------------
present? 378.235k i/100ms
blank? 375.476k i/100ms
Calculating -------------------------------------
present? 17.381M (± 7.5%) i/s - 86.238M in 5.001815s
blank? 17.877M (± 6.4%) i/s - 88.988M in 5.004634s
This improvement is mostly because those methods were hitting
`method_missing` on a lot of levels to be able to return the value.
To avoid all this stack walking we are short-circuiting those methods.
Closes#35059.
This change ensures that all query cahces are cleared across all
connections per handler for the current thread so if you write on one
connection the read will have the query cache cleared.
When I wrote the `connected_to` and `connects_to` API's I wrote them
with the idea in mind that it didn't really matter what the
handlers/roles were called as long as those connecting to the roles knew
which one wrote and which one read.
With the introduction of the middleware Rails begins to assume it's
`writing` and `reading` and there's no room for other roles. At GitHub
we've been using this method for a long time so we have a ton of legacy
code that uses different handler names `default` and `readonly`. We
could rename all our code but I think this is better for a few reasons:
- Legacy apps that have been using multiple databases for a long time
can have an eaiser time switching.
- If we later find this to cause more issues than it's worth we can
easily deprecate.
- We won't force old apps to rewrite the resolver middleware just to use
a different handler.
Adding the writing_role/reading_role required that I move the code that
creates the first handler for writing to the railtie. If I didn't move
this the core class would assign the handler before I was able to assign
a new one in my configuration and I'd end up with 3 handlers instead of
2.
Right now we only have one option that's supported, the delay. However I
can see us supporting other options in the future.
This PR refactors the options to get passed into the resolver so whether
you're using middleware or using the config options you can pass options
to the resolver. This will also make it easy to add new options in the
future.
We sometimes display simple examples of additional parameters that can be
supplied to table-wise methods like these and I found it particularly difficult
to figure out which options `t.foreign_key` accepts without drilling very deep
into the specific SchemaStatements docs.
Since it's relatively common to create foreign keys with custom column names or
primary keys, it seems like this should help quite a few people.
[ci skip]
We need to update using the timestamp from the end of the request, not
the start. For example, if a request spends 5+ seconds writing, we still
want to wait another 5 seconds for replication lag.
Since we now run the update after we yield, we need to use ensure to
make sure we update the timestamp even if there is an exception.
The following PR adds behavior to Rails to allow an application to
automatically switch it's connection from the primary to the replica.
A request will be sent to the replica if:
* The request is a read request (`GET` or `HEAD`)
* AND It's been 2 seconds since the last write to the database (because
we don't want to send a user to a replica if the write hasn't made it
to the replica yet)
A request will be sent to the primary if:
* It's not a GET/HEAD request (ie is a POST, PATCH, etc)
* Has been less than 2 seconds since the last write to the database
The implementation that decides when to switch reads (the 2 seconds) is
"safe" to use in production but not recommended without adequate testing
with your infrastructure. At GitHub in addition to the a 5 second delay
we have a curcuit breaker that checks the replication delay
and will send the query to a replica before the 5 seconds has passed.
This is specific to our application and therefore not something Rails
should be doing for you. You'll need to test and implement more robust
handling of when to switch based on your infrastructure. The auto
switcher in Rails is meant to be a basic implementation / API that acts
as a guide for how to implement autoswitching.
The impementation here is meant to be strict enough that you know how to
implement your own resolver and operations classes but flexible enough
that we're not telling you how to do it.
The middleware is not included automatically and can be installed in
your application with the classes you want to use for the resolver and
operations passed in. If you don't pass any classes into the middleware
the Rails default Resolver and Session classes will be used.
The Resolver decides what parameters define when to
switch, Operations sets timestamps for the Resolver to read from. For
example you may want to use cookies instead of a session so you'd
implement a Resolver::Cookies class and pass that into the middleware
via configuration options.
```
config.active_record.database_selector = { delay: 2.seconds }
config.active_record.database_resolver = MyResolver
config.active_record.database_operations = MyResolver::MyCookies
```
Your classes can inherit from the existing classes and reimplment the
methods (or implement more methods) that you need to do the switching.
You only need to implement methods that you want to change. For example
if you wanted to set the session token for the last read from a replica
you would reimplement the `read_from_replica` method in your resolver
class and implement a method that updates a new timestamp in your
operations class.
Previously if the `url` key in a config hash was nil we'd ignore the
configuration as invalid. This can happen when you're relying on a
`DATABASE_URL` in the env and that is not set in the environment.
```
production:
<<: *default
url: ENV['DATABASE_URL']
```
This PR fixes that case by checking if there is a `url` key in the
config instead of checking if the `url` is not nil in the config.
In addition to changing the conditional we then need to build a url hash
to merge with the original hash in the `UrlConfig` object.
Fixes#35091
The transaction used to restore fixtures is an implementation detail
that should be abstracted away. Idealy a test should behave the same
wether or not transactional fixtures are enabled.
However since transactions have been made lazy, the fixture
transaction started leaking into tests case. e.g. consider the
following (oversimplified) test:
```ruby
class SQLSubscriber
attr_accessor :sql
def initialize
@sql = []
end
def call(*, event)
sql << event[:sql]
end
end
subscriber = SQLSubscriber.new
ActiveSupport::Notifications.subscribe("sql.active_record", subscriber)
User.connection.execute('SELECT 1', 'Generic name')
assert_equal ['SELECT 1'], subscriber.sql
```
On Rails 6 it starts to break because the `sql` array will be `['BEGIN', 'SELECT 1']`.
Several things are wrong here:
- That transaction is not generated by the tested code, so it shouldn't be visible.
- The transaction is not even closed yet, which again doesn't reflect the reality.
In MySQL, the text column size is 65,535 bytes by default (1 GiB in
PostgreSQL). It is sometimes too short when people want to use a text
column, so they sometimes change the text size to mediumtext (16 MiB) or
longtext (4 GiB) by giving the `limit` option.
Unlike MySQL, PostgreSQL doesn't allow the `limit` option for a text
column (raises ERROR: type modifier is not allowed for type "text").
So `limit: 4294967295` (longtext) couldn't be used in Action Text.
I've allowed changing text and blob size without giving the `limit`
option, it prevents that migration failure on PostgreSQL.
While working on another feature for multiple databases (auto-switching)
I observed that in development the first request won't autoload the
application record connection for the primary database and may not yet
know about the replica connection.
In my test application this caused the application to thrown an error if
I tried to send the first request to the replica before the replica was
connected. This wouldn't be an issue in production because the
application is preloaded.
In order to fix this I decided to leave the original error message and
delete the new error message. I updated the original error message to
include the `role` to make it a bit clearer that the connection isn't
established for that particular role.
The error now reads:
```
No connection pool with 'primary' found for the 'reading' role.
```
A single database application will continue uisng the original error
message:
```
No connection pool with 'primary' found.
```
Allows aliasing, predications, ordering, and various other functions on `And` and `Case` nodes. This brings them in line with other nodes like `Binary` and `Unary`.
Currently `conn.column_exists?("testings", "created_at", "datetime")`
returns false even if the table has the `created_at` column.
That reason is that `column.type` is a symbol but passed `type` is not
normalized to symbol unlike `column_name`, it is surprising behavior to
me.
I've improved that to normalize a value before comparison.
Since #31230, `change_column` is executed as a bulk statement.
That caused incorrect type casting column default by looking up the
before changed type, not the after changed type.
In a bulk statement, we can't use `change_column_default_for_alter` if
the statement changes the column type.
This fixes the type casting to use the constructed target sql_type.
Fixes#34938.
Since 31ffbf8d, finder methods no longer raise `RangeError`. So
`StatementCache#execute` is the only place to raise the exception for
finder queries.
`StatementCache` is used for simple equality queries in the codebase.
This means that if `StatementCache#execute` raises `RangeError`, the
result could always be regarded as empty.
So `StatementCache#execute` just return nil in that range error case,
and treat that as empty in the caller side, then we can avoid catching
the exception in much places.
Currently several queries cannot return correct result due to incorrect
`RangeError` handling.
First example:
```ruby
assert_equal true, Topic.where(id: [1, 9223372036854775808]).exists?
assert_equal true, Topic.where.not(id: 9223372036854775808).exists?
```
The first example is obviously to be true, but currently it returns
false.
Second example:
```ruby
assert_equal topics(:first), Topic.where(id: 1..9223372036854775808).find(1)
```
The second example also should return the object, but currently it
raises `RecordNotFound`.
It can be seen from the examples, the queries including large number
assuming empty result is not always correct.
Therefore, This change handles `RangeError` to generate executable SQL
instead of raising `RangeError` to users to always return correct
result. By this change, it is no longer raised `RangeError` to users.
When we added support for multiple databases through a 3-tiered config
and configuration objects this error message got a bit convoluted.
Previously if you had an application with a missing configuation and
multiple databases the error message would look like this:
```
'doesnexist' database is not configured. Available: development,
development, test, test, production, production
(ActiveRecord::AdapterNotSpecified)
```
That's not very descriptive since it duplicates the environments
(because there are multiple databases per environment for this
application).
To fix this I've constructed a bit more readable error message which now
reads like this if you have a multi db app:
```
The `doesntexist` database is not configured for the `production`
environment. (ActiveRecord::AdapterNotSpecified)
Available databases configurations are:
development: primary, primary_readonly
test: primary, primary_readonly
production: primary, primary_readonly
```
And like this if you have a single db app:
```
The `doesntexist` database is not configured for the `production`
environment. (ActiveRecord::AdapterNotSpecified)
Available databases configurations are:
development
test
```
This makes the error message more readable and presents the user all
available options for the database connections.
We need this in order to be able to add this migration for users that
use ActiveStorage during update their apps from Rails 5.2 to Rails 6.0.
Related to #33405
`rake app:update` should update active_storage
`rake app:update` should execute `rake active_storage:update`
if it is used in the app that is being updated.
It will add new active_storage's migrations to users' apps during update Rails.
Context https://github.com/rails/rails/pull/33405#discussion_r204239399
Also, see a related discussion in the Campfire:
https://3.basecamp.com/3076981/buckets/24956/chats/12416418@1236713081
This PR addresses the issue described in #28025. On `dependent: :nullify` strategy only the foreign key of the relation is nullified. However on polymorphic associations the `*_type` column is not nullified leaving the record with a NULL `*_id` but the `*_type` column is present.
This attr writer was introduced at 7db90aa, but the usage is already
removed at bd2f5c0 before v3.2.0.rc1 is released.
If we'd like to customize the visitor in the connection, `arel_visitor`
which is implemented in all adapters (mysql2, postgresql, sqlite3,
oracle-enhanced, sqlserver) could be used for the purpose #23515.
This commit adds support for endless ranges, e.g. (1..), that were added
in Ruby 2.6. They're functionally equivalent to explicitly specifying
Float::INFINITY as the end of the range.
* Enable `Lint/UselessAssignment` cop to avoid unused variable warnings
Since we've addressed the warning "assigned but unused variable"
frequently.
370537de053040446cec5ed618e19276ebafe594
And also, I've found the unused args in c1b14ad which raises no warnings
by the cop, it shows the value of the cop.
Since Ruby 2.6.0 NilClass#to_d is returning `BigDecimal` 0.0, this
breaks `average` compatibility with prior Ruby versions. This patch
makes `average` return `nil` in all Ruby versions when there are no
rows.
`nil`, `Numeric`, and `String` are most basic objects which are passed
to `type_cast`. But now each `when *types_which_need_no_typecasting`
evaluation allocates extra two arrays, it makes `type_cast` slower.
The `types_which_need_no_typecasting` was introduced at #15351, but the
method isn't useful (never used any adapters) since all adapters
(sqlite3, mysql2, postgresql, oracle-enhanced, sqlserver) still
overrides the `_type_cast`.
Just expanding the method would make the `type_cast` 2x faster.
```ruby
module ActiveRecord
module TypeCastFast
def type_cast_fast(value, column = nil)
value = id_value_for_database(value) if value.is_a?(Base)
if column
value = type_cast_from_column(column, value)
end
_type_cast_fast(value)
rescue TypeError
to_type = column ? " to #{column.type}" : ""
raise TypeError, "can't cast #{value.class}#{to_type}"
end
private
def _type_cast_fast(value)
case value
when Symbol, ActiveSupport::Multibyte::Chars, Type::Binary::Data
value.to_s
when true then unquoted_true
when false then unquoted_false
# BigDecimals need to be put in a non-normalized form and quoted.
when BigDecimal then value.to_s("F")
when nil, Numeric, String then value
when Type::Time::Value then quoted_time(value)
when Date, Time then quoted_date(value)
else raise TypeError
end
end
end
end
conn = ActiveRecord::Base.connection
conn.extend ActiveRecord::TypeCastFast
Benchmark.ips do |x|
x.report("type_cast") { conn.type_cast("foo") }
x.report("type_cast_fast") { conn.type_cast_fast("foo") }
x.compare!
end
```
```
Warming up --------------------------------------
type_cast 58.733k i/100ms
type_cast_fast 101.364k i/100ms
Calculating -------------------------------------
type_cast 708.066k (± 5.9%) i/s - 3.583M in 5.080866s
type_cast_fast 1.424M (± 2.3%) i/s - 7.197M in 5.055860s
Comparison:
type_cast_fast: 1424240.0 i/s
type_cast: 708066.0 i/s - 2.01x slower
```
In an application that has a primary and replica database the data
inserted on the primary connection will not be able to be read by the
replica connection.
In a test like this:
```
test "creating a home and then reading it" do
home = Home.create!(owner: "eileencodes")
ActiveRecord::Base.connected_to(role: :default) do
assert_equal 3, Home.count
end
ActiveRecord::Base.connected_to(role: :readonly) do
assert_equal 3, Home.count
end
end
```
The home inserted in the beginning of the test can't be read by the
replica database because when the test is started a transasction is
opened byy `setup_fixtures`. That transaction remains open for the
remainder of the test until we are done and run `teardown_fixtures`.
Because the data isn't actually committed to the database the replica
database cannot see the data insertion.
I considered a couple ways to fix this. I could have written a database
cleaner like class that would allow the data to be committed and then
clean up that data afterwards. But database cleaners can make the
database slow and the point of the fixtures is to be fast.
In GitHub we solve this by sharing the connection pool for the replicas
with the primary (writing) connection. This is a bit hacky but it works.
Additionally since we define `replica? || preventing_writes?` as the
code that blocks writes to the database this will still prevent writing
on the replica / readonly connection. So we get all the behavior of
multiple connections for the same database without slowing down the
database.
In this PR the code loops through the handlers. If the handler doesn't
match the default handler then it retrieves the connection pool from the
default / writing handler and assigns the reading handler's connections
to that pool.
Then in enlist_fixture_connections it maps all the connections for the
default handler because all the connections are now available on that
handler so we don't need to loop through them again.
The test uses a temporary connection pool so we can test this with
sqlite3_mem. This adapter doesn't behave the same as the others and
after looking over how the query cache test works I think this is the
most correct. The issues comes when calling `connects_to` because that
establishes new connections and confuses the sqlite3_mem adapter. I'm
not entirely sure why but I wanted to make sure we tested all adapters
for this change and I checked that it wasn't the shared connection code
that was causing issues - it was the `connects_to` code.
That ability was introduced at #11898 as `Relation#update` without
giving ids, so the ability on the class level is not documented and not
tested.
c83e30d which fixes#33470 has lost two undocumented abilities.
One has fixed at 5c65688, but I missed the ability on the class level.
Removing any feature should not be suddenly happened in a stable version
even if that is not documented.
I've restored the ability and added test case to avoid any regression in
the future.
Fixes#34743.
This reverts 27c6c07 since `arel_attr.to_s` is not right way to avoid
the type error.
That to_s returns `"#<struct Arel::Attributes::Attribute ...>"`, there
is no reason to match the regex to the inspect form.
And also, the regex path is not covered by our test cases. I've tweaked
the regex for redundant part and added assertions for the regex path.
Since `migration_context` has `migrations_paths` itself and provides
methods which returning values from parsed migration files, so there is
no reason to use the `parse_migration_filename` low level API directly.
If you try to call `connected_to` with a role that doesn't have an
established connection you used to get an error that said:
```
>> ActiveRecord::Base.connected_to(role: :i_dont_exist) { Home.first }
ActiveRecord::ConnectionNotEstablished Exception: No connection pool
with 'primary' found.
```
This is confusing because the connection could be established but we
spelled the role wrong.
I've changed this to raise if the `role` used in `connected_to` doesn't
have an associated handler. Users who encounter this should either check
that the role is spelled correctly (writin -> writing), establish a
connection to that role in the model with connects_to, or use the
`database` keyword for the `role`.
I think this will provide a less confusing error message for those
starting out with multiple databases.
Thanks to ko1, passing block parameter to another method is
significantly optimized in Ruby 2.5.
https://bugs.ruby-lang.org/issues/14045
Thus we no longer need to keep this ugly hack.
Since MySQL 5.7.9, the `innodb_default_row_format` option defines the
default row format for InnoDB tables. The default setting is `DYNAMIC`.
The row format is required for indexing on `varchar(255)` with `utf8mb4`
columns.
As long as using MySQL 5.6, CI won't be passed even if MySQL server
setting is properly configured the same as MySQL 5.7
(`innodb_file_per_table = 1`, `innodb_file_format = 'Barracuda'`, and
`innodb_large_prefix = 1`) since InnoDB table is created as the row
format `COMPACT` by default on MySQL 5.6, therefore indexing on string
with `utf8mb4` columns aren't succeeded.
Making `ROW_FORMAT=DYNAMIC` create table option by default for legacy
MySQL version would mitigate the indexing issue on the user side, and it
makes CI would be passed on MySQL 5.6 which is configured properly.
BEGIN transaction would cause COMMIT or ROLLBACK, so unless COMMIT and
ROLLBACK aren't treated as write queries as well as BEGIN, the
`ReadOnlyError` would be raised.
I originally named this `StatementInvalid` because that's what we do in
GitHub, but `@tenderlove` pointed out that this means apps can't test
for or explitly rescue this error. `StatementInvalid` is pretty broad so
I've renamed this to `ReadOnlyError`.
Unlike the `Relation#delete_all`, `delete_all` on collection proxy
doesn't return affected count. Since the `CollectionProxy` is a subclass
of the `Relation`, this inconsistency is probably not intended, so it
should return the count consistently.
This PR adds the ability to prevent writes to a database even if the
database user is able to write (ie the database is a primary and not a
replica).
This is useful for a few reasons: 1) when converting your database from
a single db to a primary/replica setup - you can fix all the writes on
reads early on, 2) when we implement automatic database switching or
when an app is manually switching connections this feature can be used
to ensure reads are reading and writes are writing. We want to make sure
we raise if we ever try to write in read mode, regardless of database
type and 3) for local development if you don't want to set up multiple
databases but do want to support rw/ro queries.
This should be used in conjunction with `connected_to` in write mode.
For example:
```
ActiveRecord::Base.connected_to(role: :writing) do
Dog.connection.while_preventing_writes do
Dog.create! # will raise because we're preventing writes
end
end
ActiveRecord::Base.connected_to(role: :reading) do
Dog.connection.while_preventing_writes do
Dog.first # will not raise because we're not writing
end
end
```
Follow up #33394.
#33394 only fixes the case of scoping with klass methods in the scope
block which invokes `klass.all`.
Query methods in the scope block also need to invoke `klass.all` to be
affected by the scoping.
Before this patch, singular and collection associations
had different implementations of the #find_target method.
This patch reuses the code properly through extending the low level
methods.
When calling ordered finder methods such as +first+ or +last+ without an
explicit order clause, ActiveRecord sorts records by primary key. This
can result in unpredictable and surprising behaviour when the primary
key is not an auto-incrementing integer, for example when it's a UUID.
This change makes it possible to override the column used for implicit
ordering such that +first+ and +last+ will return more predictable
results. For Example:
class Project < ActiveRecord::Base
self.implicit_order_column = "created_at"
end
https://www.postgresql.org/support/versioning/
- 9.1 EOLed on September 2016.
- 9.2 EOLed on September 2017.
9.3 is also not supported since Nov 8, 2018. https://www.postgresql.org/about/news/1905/
I think it may be a little bit early to drop PostgreSQL 9.3 yet.
* Deprecated `supports_ranges?` since no other databases support range data type
* Add `supports_materialized_views?` to abstract adapter
Materialized views itself is supported by other databases, other connection adapters may support them
* Remove `with_manual_interventions`
It was only necessary for PostgreSQL 9.1 or earlier
* Drop CI against PostgreSQL 9.2
Before:
```
LOG: execute <unnamed>: SELECT t.oid, t.typname
FROM pg_type as t
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool')
LOG: execute <unnamed>: SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype
FROM pg_type as t
LEFT JOIN pg_range as r ON oid = rngtypid
WHERE
t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'interval', 'path', 'line', 'polygon', 'circle', 'lseg', 'box', 'time', 'timestamp', 'numeric')
OR t.typtype IN ('r', 'e', 'd')
OR t.typinput::varchar = 'array_in'
OR t.typelem != 0
LOG: statement: SHOW TIME ZONE
LOG: statement: SELECT 1
LOG: execute <unnamed>: SELECT COUNT(*)
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r','v','m') -- (r)elation/table, (v)iew, (m)aterialized view
AND c.relname = 'accounts'
AND n.nspname = ANY (current_schemas(false))
```
After:
```
LOG: execute <unnamed>: SELECT t.oid, t.typname
FROM pg_type as t
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool')
LOG: execute <unnamed>: SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype
FROM pg_type as t
LEFT JOIN pg_range as r ON oid = rngtypid
WHERE
t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'interval', 'path', 'line', 'polygon', 'circle', 'lseg', 'box', 'time', 'timestamp', 'numeric')
OR t.typtype IN ('r', 'e', 'd')
OR t.typinput::varchar = 'array_in'
OR t.typelem != 0
LOG: statement: SHOW TIME ZONE
LOG: statement: SELECT 1
LOG: execute <unnamed>: SELECT COUNT(*)
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r','v','m') -- (r)elation/table, (v)iew, (m)aterialized view
AND c.relname = 'accounts'
AND n.nspname = ANY (current_schemas(false))
```
Currently the query cache is only aware of one handler so once we added
multiple databases switching on the handler we broke query cache for
those reading connections.
While #34054 is the proper fix, that fix is not straight forward and I
want to make sure that the query cache isn't just broken for all other
connections not in the main handler.
The connection handler was using the RuntimeRegistry which kind of
implies it's a per thread registry. But it's actually per fiber.
If you have an application that uses fibers and you're using multiple
databases, when you switch the connection handler to swap connections
new fibers running on the same thread used to get a different connection
id. This PR changes the code to actually use a thread so that we get
the same connection.
Fixes https://github.com/rails/rails/issues/30047
[Eileen M. Uchitelle, Aaron Patterson, & Arthur Neeves]
Since both methods are public API I think it makes sense to add these tests
in order to prevent any regression in the behavior of those methods after the 6.0 release.
Exercise `connected_to`
- Ensure that the method raises with both `database` and `role` arguments
- Ensure that the method raises without `database` and `role`
Exercise `connects_to`
- Ensure that the method returns an array of established connections(as mentioned
in the docs of the method)
Related to #34052
mysql has a great implementation to suppress multiple parens for union
sql statements.
This moves that functionality to the generic implementation
This also introduces that functionality for UNION ALL
This commit adds support for the
`ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.create_unlogged_tables`
setting, which turns `CREATE TABLE` SQL statements into
`CREATE UNLOGGED TABLE` statements.
This can improve PostgreSQL performance but at the
cost of data durability, and thus it is highly recommended
that you *DO NOT* enable this in a production environment.
The test added in 12b0b26df7 passes even
without this code since 9b8c7796a9, as the
call to `id` in `remember_transaction_record_state` now triggers a
`sync_with_transaction_state` which discards the leftover state from the
previous transaction.
This issue had already been fixed for `save!`, `destroy` and `touch` in
caae79a385, but continued to affect `save`
because the call to `rollback_active_record_state!` in that method would
increment the transaction level before `add_to_transaction` could clear
it, preventing the fix from working correctly.
As `rollback_active_record_state!` was removed entirely in
48007d5390, this code is no longer needed.
When a record with transactional callbacks is saved, it's attached to
the current transaction so that the callbacks can be run when the
transaction is committed. Records can also be added manually with
`add_transaction_record`, even if they have no transactional callbacks.
When a nested transaction is committed, its records are transferred to
the parent transaction, as transactional callbacks should only be run
when the outermost transaction is committed (the "real" transaction).
However, this currently only happens when the record has transactional
callbacks, and not when added manually with `add_transaction_record`.
If a record is added to a nested transaction, we should always attach it
to the parent transaction when the nested transaction is committed,
regardless of whether it has any transactional callbacks.
[Eugene Kenny & Ryuta Kamizono]
There was a bug in the handling of duplicates when
assigning (replacing) associated records, which made the result
dependent on whether a given record was associated already before
being assigned anew. E.g.
post.people = [person, person]
post.people.count
# => 2
while
post.people = [person]
post.people = [person, person]
post.people.count
# => 1
This change adds a test to provoke the former incorrect behavior, and
fixes it.
Cause of the bug was the handling of record collections as sets, and
using `-` (difference) and `&` (union) operations on them
indiscriminately. This temporary conversion to sets would eliminate
duplicates.
The fix is to decorate record collections for these operations, and
only for the `has_many :through` case. It is done by counting
occurrences, and use the record together with the occurrence number as
element, in order to make them work well in sets. Given
a, b = *Person.all
then the collection used for finding the difference or union of
records would be internally changed from
[a, b, a]
to
[[a, 1], [b, 1], [a, 2]]
for these operations. So a first occurrence and a second occurrence
would be distinguishable, which is all that is necessary for this
task.
Fixes#33942.
The `read_attribute` method always returns the primary key when asked to
read the `id` attribute, even if the primary key isn't named `id`, and
even if another attribute named `id` exists.
For the `inspect`, `attribute_for_inspect` and `pretty_print` methods,
this behaviour is undesirable, as they're used to examine the internal
state of the record. By using `_read_attribute` instead, we'll get the
real value of the `id` attribute.
The `@connection` is no longer used since ee5ab22.
Originally the `@connection` was useless because it is only used in
`timestamp_column_names`, which is only used if `model_class` is given.
If `model_class` is given, the `@connection` is always
`model_class.connection`.
Previosly, `update_columns` would just take whatever keys you gave it
and tried to run the update query. Most likely this would result in an
error from the database. However, if the column actually did exist, but
was in `ignored_columns`, this would result in the method returning
successfully when it should have raised, and an attribute that should
not exist written to `@attributes`.
When loading fixtures, Ruby 1.9's hash key ordering means that HABTM
join table rows are always loaded before the parent table rows,
violating foreign key constraints that may be in place. This very
simple change ensures that the parent table's key appears first in the
hash. Violations may still occur if fixtures are loaded in the wrong
order but those instances can be avoided unlike this one.
At fc0e3354af,
```rb
relation = relation.where(conditions)
```
was rewritten to:
```rb
relation.where!(condition)
```
This change accidentally changed the result of `Topic.exists?({})` from true to false.
To fix this regression, first I moved the blank check logic (`opts.blank?`) from `#where` to `#where!`,
because I thought `#where!` should be identical to `#where`, except that instead of returning a new relation,
it adds the condition to the existing relation.
But on second thought after some discussion on https://github.com/rails/rails/pull/34329,
I started to think that just fixing `#construct_relation_for_exists` is more preferable
than changing `#where` and `#where!`.
Since #33844, eager loading/preloading with too many and/or too large
ids won't be broken by pre-checking whether the value is constructable
or not.
But the pre-checking caused the type to be evaluated at relation build
time instead of at the query execution time, that is breaking an
expectation for some apps.
I've made the pre-cheking lazy as much as possible, that is no longer
happend at relation build time.
when user has no parent table access privileges
Refer https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-13.html#mysqld-8-0-13-errors
>>
* Previously, the ER_NO_REFERENCED_ROW_2 and ER_ROW_IS_REFERENCED_2 error messages
for foreign key operations were displayed and revealed information about parent tables,
even when the user had no parent table access privileges. Error handling for this situation has been revised:
* If the user does have table-level privileges for all parent tables,
ER_NO_REFERENCED_ROW_2 and ER_ROW_IS_REFERENCED_2 are displayed, the same as before.
* If the user does not have table-level privileges for all parent tables,
more generic error messages are displayed instead (ER_NO_REFERENCED_ROW and ER_ROW_IS_REFERENCED).
<<
This pull request addresses these 3 failures:
```ruby
$ ARCONN=mysql2 bundle exec ruby -w -Itest test/cases/adapter_test.rb -n /foreign/
Using mysql2
Run options: -n /foreign/ --seed 14251
F
Failure:
ActiveRecord::AdapterForeignKeyTest#test_foreign_key_violations_are_translated_to_specific_exception_with_validate_false [test/cases/adapter_test.rb:348]:
[ActiveRecord::InvalidForeignKey] exception expected, not
Class: <ActiveRecord::StatementInvalid>
Message: <"Mysql2::Error: Cannot add or update a child row: a foreign key constraint fails: INSERT INTO `fk_test_has_fk` (`fk_id`) VALUES (1231231231)">
... snip ...
rails test test/cases/adapter_test.rb:343
F
Failure:
ActiveRecord::AdapterForeignKeyTest#test_foreign_key_violations_on_delete_are_translated_to_specific_exception [test/cases/adapter_test.rb:368]:
[ActiveRecord::InvalidForeignKey] exception expected, not
Class: <ActiveRecord::StatementInvalid>
Message: <"Mysql2::Error: Cannot delete or update a parent row: a foreign key constraint fails: DELETE FROM fk_test_has_pk WHERE pk_id = 1">
... snip ...
rails test test/cases/adapter_test.rb:365
F
Failure:
ActiveRecord::AdapterForeignKeyTest#test_foreign_key_violations_on_insert_are_translated_to_specific_exception [test/cases/adapter_test.rb:358]:
[ActiveRecord::InvalidForeignKey] exception expected, not
Class: <ActiveRecord::StatementInvalid>
Message: <"Mysql2::Error: Cannot add or update a child row: a foreign key constraint fails: INSERT INTO fk_test_has_fk (fk_id) VALUES (0)">
... snip ...
rails test test/cases/adapter_test.rb:357
Finished in 0.087370s, 34.3366 runs/s, 34.3366 assertions/s.
3 runs, 3 assertions, 3 failures, 0 errors, 0 skips
$
```
When `attr_name` is passed as a symbol, it's currently converted to a
string by `attribute_alias?`, and potentially also `attribute_alias`,
as well as by the `read_attribute`/`write_attribute` method itself.
By converting `attr_name` to a string up front, the extra allocations
related to attribute aliases can be avoided.
AR instance support `filter_parameters` since #33756.
Though Regex or Proc is valid as `filter_parameters`,
they are not supported as AR#inspect.
I also add :mask option and #filter_params to
`ActiveSupport::ParameterFilter#new` to implement this.
When an `updated_at` column exists on the model, but is not available on the instance (likely due to a select), we should raise an error rather than silently not generating a cache_version. Without this behavior it's likely that cache entries will not be able to be invalidated and this will happen without notice.
This behavior was reported and described by @lsylvester in https://github.com/rails/rails/pull/34197#issuecomment-429668759.
Currently, the `updated_at` field is used to generate a `cache_version`. Some database adapters return this timestamp value as a string that must then be converted to a Time value. This process requires a lot of memory and even more CPU time. In the case where this value is only being used for a cache version, we can skip the Time conversion by using the string value directly.
- This PR preserves existing cache format by converting a UTC string from the database to `:usec` format.
- Some databases return an already converted Time object, in those instances, we can directly use `created_at`.
- The `updated_at_before_type_cast` can be a value that comes from either the database or the user. We only want to optimize the case where it is from the database.
- If the format of the cache version has been changed, we cannot apply this optimization, and it is skipped.
- If the format of the time in the database is not UTC, then we cannot use this optimization, and it is skipped.
Some databases (notably PostgreSQL) returns a variable length nanosecond value in the time string. If the value ends in a zero, then it is truncated For instance instead of `2018-10-12 05:00:00.000000` the value `2018-10-12 05:00:00` is returned. We detect this case and pad the remaining zeros to ensure consistent cache version generation.
Before: Total allocated: 743842 bytes (6626 objects)
After: Total allocated: 702955 bytes (6063 objects)
(743842 - 702955) / 743842.0 # => 5.4% ⚡️⚡️⚡️⚡️⚡️
Using the CodeTriage application and derailed benchmarks this PR shows between 9-11% (statistically significant) performance improvement versus the commit before it.
Special thanks to @lsylvester for helping to figure out a way to preserve the usec format and for helping with many implementation details.
`init_with` and `init_from_db` are almost the same code except decode
`coder`.
And also, named `init_from_db` is a little misreading, a raw values hash
from the database is already converted to an attributes object by
`attributes_builder.build_from_database`, so passed `attributes` in that
method is just an attributes object.
I renamed that method to `init_with_attributes` since the method is
shared with `init_with` to initialize an empty model object.
Will allow sub classes to override the protected
`#check_version` method hook if desired.
For example, this will be most helpful in sub classes that wish
to support lazy initialization because the version check can
be postponed until the connection is ready to be initialized.
This is an alternative to https://github.com/rails/rails/pull/34195
The active record `respond_to?` method needs to do two things if `super` does not say that the method exists. It has to see if the "name" being passed in represents a column in the table. If it does then it needs to pass it to `has_attribute?` to see if the key exists in the current object. The reason why this is slow is that `has_attribute?` needs a string and most (almost all) objects passed in are symbols.
The only time we need to allocate a string in this method is if the column does exist in the database, and since these are a limited number of strings (since column names are a finite set) then we can pre-generate all of them and use the same string.
We generate a list hash of column names and convert them to symbols, and store the value as the string name. This allows us to both check if the "name" exists as a column, but also provides us with a string object we can use for the `has_attribute?` call.
I then ran the test suite and found there was only one case where we're intentionally passing in a string and changed it to a symbol. (However there are tests where we are using a string key, but they don't ship with rails).
As re-written this method should never allocate unless the user passes in a string key, which is fairly uncommon with `respond_to?`.
This also eliminates the need to special case every common item that might come through the method via the `case` that was originally added in f80aa59946 (by me) and then with an attempt to extend in https://github.com/rails/rails/pull/34195.
As a bonus this reduces 6,300 comparisons (in the CodeTriage app homepage) to 450 as we also no longer need to loop through the column array to check for an `include?`.
Ruby uses the original method name, so will show the __temp__ method
name in the backtrace. However, in the common case the method name
is compatible with the `def` keyword, so we can avoid the __temp__
method name in that case to improve the name shown in backtraces
or TracePoint#method_id.
This PR adds the ability to 1) connect to multiple databases in a model,
and 2) switch between those connections using a block.
To connect a model to a set of databases for writing and reading use
the following API. This API supercedes `establish_connection`. The
`writing` and `reading` keys represent handler / role names and
`animals` and `animals_replica` represents the database key to look up
the configuration hash from.
```
class AnimalsBase < ApplicationRecord
connects_to database: { writing: :animals, reading: :animals_replica }
end
```
Inside the application - outside the model declaration - we can switch
connections with a block call to `connected_to`.
If we want to connect to a db that isn't default (ie readonly_slow) we
can connect like this:
Outside the model we may want to connect to a new database (one that is
not in the default writing/reading set) - for example a slow replica for
making slow queries. To do this we have the `connected_to` method that
takes a `database` hash that matches the signature of `connects_to`. The
`connected_to` method also takes a block.
```
AcitveRecord::Base.connected_to(database: { slow_readonly: :primary_replica_slow }) do
ModelInPrimary.do_something_thats_slow
end
```
For models that are already loaded and connections that are already
connected, `connected_to` doesn't need to pass in a `database` because
you may want to run queries against multiple databases using a specific
role/handler.
In this case `connected_to` can take a `role` and use that to swap on
the connection passed. This simplies queries - and matches how we do it
in GitHub. Once you're connected to the database you don't need to
re-connect, we assume the connection is in the pool and simply pass the
handler we'd like to swap on.
```
ActiveRecord::Base.connected_to(role: :reading) do
Dog.read_something_from_dog
ModelInPrimary.do_something_from_model_in_primary
end
```
When defining a Hash enum it can be easy to use [] instead of {}. This
commit checks that only valid definition values are provided, those can
be a Hash, an array of Symbols or an array of Strings. Otherwise it
raises an ArgumentError.
Fixes#33961
Before:
```
Pet Destroy (0.8ms) DELETE FROM `pets` WHERE `pets`.`pet_id` IN (SELECT `pet_id` FROM (SELECT DISTINCT `pets`.`pet_id` FROM `pets` LEFT OUTER JOIN `toys` ON `toys`.`pet_id` = `pets`.`pet_id` WHERE `toys`.`name` = ?) AS __active_record_temp) [["name", "Bone"]]
```
After:
```
Pet Destroy (1.0ms) DELETE `pets` FROM `pets` LEFT OUTER JOIN `toys` ON `toys`.`pet_id` = `pets`.`pet_id` WHERE `toys`.`name` = ? [["name", "Bone"]]
```
The delegation methods to named scope are defined when `method_missing`
is invoked on the relation.
Since #29301, the receiver in the named scope is changed to the relation
like others (e.g. `default_scope`, etc) for consistency.
Most named scopes would be delegated from relation by `method_missing`,
since we don't allow scopes to be defined which conflict with instance
methods on `Relation` (#31179). But if a named scope is defined with the
same name as any method on the `superclass` (e.g. `Kernel.open`), the
`method_missing` on the relation is not invoked.
To address the issue, make the delegation methods to named scope is
generated in the definition time.
Fixes#34098.
Aaron Patterson