Commit Graph

14399 Commits

Author SHA1 Message Date
Ryuta Kamizono 6a1a1e66ea Use placeholder for `type_condition` predicate
Before:

```
SELECT "comments".* FROM "comments" WHERE "comments"."type" IN ('VerySpecialComment') AND "comments"."post_id" = ? LIMIT ?  [["post_id", 4], ["LIMIT", 1]]
```

After:

```
SELECT "comments".* FROM "comments" WHERE "comments"."type" = ? AND "comments"."post_id" = ? LIMIT ?  [["type", "VerySpecialComment"], ["post_id", 4], ["LIMIT", 1]]
```
2019-02-18 01:39:06 +09:00
Ryuta Kamizono 49bcb008cb Fix eager loading polymorphic association with mixed table conditions
This fixes a bug that the `foreign_key` and the `foreign_type` are
separated as different table conditions if a polymorphic association has
a scope that joins another tables.
2019-02-18 00:41:43 +09:00
Ryuta Kamizono 5ca19efafe
Merge pull request #35274 from AlexBrodianoi/fix_does_not_support_reverse
Raise ActiveRecord::IrreversibleOrderError if nulls first/last is not a single ordering argument.
2019-02-17 12:39:44 +09:00
Finn Young db930ec0fd Raise ActiveRecord::IrreversibleOrderError if nulls first/last is not a single ordering argument. 2019-02-17 00:41:30 +00:00
Ryuta Kamizono 311f001167 Fix `order` with custom attributes
This follows up 0ee96d13de.
2019-02-17 02:44:37 +09:00
Ryuta Kamizono 3ac195c534 Fix the regex that extract mismatched foreign key information
The CI failure for `test_errors_for_bigint_fks_on_integer_pk_table` is
due to the poor regex that extract all ``` `(\w+)` ``` like parts from
the message (`:foreign_key` should be `"old_car_id"`, but `"engines"`):

https://travis-ci.org/rails/rails/jobs/494123455#L1703

I've improved the regex more strictly and have more exercised mismatched
foreign key tests.

Fixes #35294
2019-02-17 00:53:26 +09:00
Ryuta Kamizono fed7888c83 Merge pull request #35297 from yhara/fix-ar-connection-handler-leak
Fix possible memory leak of ConnectionHandler
2019-02-16 23:11:27 +09:00
Yutaka HARA 20a7262790 Fix possible memory leak of ConnectionHandler
refs #35296
2019-02-16 22:31:15 +09:00
Ryuta Kamizono 6572175d4d Refactor `remove_foreign_key` to delete the foreign key before `alter_table` 2019-02-16 14:56:29 +09:00
Ryuta Kamizono b353391846 Merge pull request #35286 from matthewdunbar/master
Properly handle cached queries with too many bind parameters
2019-02-16 14:16:44 +09:00
Ryuta Kamizono 4c6171d605 Deprecate using class level querying methods if the receiver scope regarded as leaked
This deprecates using class level querying methods if the receiver scope
regarded as leaked, since #32380 and #35186 may cause that silently
leaking information when people upgrade the app.

We need deprecation first before making those.
2019-02-15 17:40:15 +09:00
Matthew Dunbar 795c0f8205 Properly handle cached queries with too many bind parameters 2019-02-14 23:25:44 -05:00
Ryuta Kamizono cdb8697b4a Revert "Merge pull request #35186 from kamipo/fix_leaking_scope_on_relation_create"
This reverts commit b67d5c6ded, reversing
changes made to 2e018361c7.

Reason: #35186 may cause that silently leaking information when people
upgrade the app.

We need deprecation first before making this.
2019-02-15 12:06:45 +09:00
Abhay Nikam 85efb7cfe6 Minor changes to deprecation warning message after 35242 2019-02-15 00:33:36 +05:30
Eileen M. Uchitelle 5f7180676b
Merge pull request #35242 from eileencodes/add-setter-and-deprecation-for-configurations-hashes
Add setter and deprecation for configurations hashes
2019-02-14 12:58:54 -05:00
eileencodes 06f9434342 Improve errors and handling of hashes for database configurations
In chat Sam Saffron asked how to use the setter now that configurations
is no longer a hash and you need to do AR::Base.configurations["test"]=.

Technically you can do `ActiveRecord::Base.configurations = { the hash
}` but I realized the old way throws an error and is unintuitive.

To aid in the transition from hashes to objects this PR makes a few
changes:

1) Re-adds a deprecated hash setter `[]=` that will add a new hash
to the configurations list OR replace an existing hash if that
environment is already present. This won't be supported in future Rails
versions but a good error is important.

2) Changed to throw deprecation warnings on the methods we decided to support
for hash conversion and raise on the methods we don't support.

3) Refactored the setter/getter hash deprecation warnings messages and
rewrote them.

Getters message:

```
DEPRECATION WARNING: `ActiveRecord::Base.configurations` no longer
returns a hash. Methods that act on the hash like `values` are
deprecated and will be removed in Rails 6.1. Use the `configs_for`
method to collect and iterate over the database configurations.
```

Setter message:

```
DEPRECATION WARNING: Setting `ActiveRecord::Base.configurations` with
`[]=` is deprecated. Use `ActiveRecord::Base.configurations=` directly
to set the configurations instead.
```

4) Rewrote the legacy configurations test file to test all the public
methods in the DatabaseConfigurations class.
2019-02-14 08:25:52 -05:00
Ryuta Kamizono 4f2a635661 Revert "Chaining named scope is no longer leaking to class level querying methods"
This reverts #32380, since this may cause that silently leaking
information when people upgrade the app.

We need deprecation first before making this.
2019-02-14 05:22:41 +09:00
Eileen M. Uchitelle be099cc13f
Merge pull request #35244 from palkan/fix/activerecord-database-uri-resolve
Fix database configurations building when DATABASE_URL present
2019-02-13 10:57:05 -05:00
Ryuta Kamizono 47e3bbeb90 Revert "Merge pull request #35127 from bogdan/counter-cache-loading"
This reverts commit eec3e28a1a, reversing
changes made to 5588fb4802.

Reason: Marking as loaded without actual loading is too greedy optimization.

See more context #35239.

Closes #35239.

[Edouard CHIN & Ryuta Kamizono]
2019-02-13 22:45:35 +09:00
utilum 87a5379b42 Ruby 2.7 warning: creating a Proc without a block
As of [Revision 66772](
https://bugs.ruby-lang.org/projects/ruby-trunk/repository/trunk/revisions/66772)
 `Proc.new` without giving a block emits `warning: tried to create Proc object without a block`.

This commit fixes cases where Rails test suit tickles this warning.

See CI logs:
https://travis-ci.org/rails/rails/jobs/487205819#L1161-L1190
https://travis-ci.org/rails/rails/jobs/487205821#L1154-1159
https://travis-ci.org/rails/rails/jobs/487205821#L1160-L1169
https://travis-ci.org/rails/rails/jobs/487205821#L1189
https://travis-ci.org/rails/rails/jobs/487254404#L1307-L1416
https://travis-ci.org/rails/rails/jobs/487254405#L1174-L1191
2019-02-13 02:11:21 +01:00
Vladimir Dementyev d9b261a340
Fix database configurations building when DATABASE_URL present 2019-02-12 17:00:17 -05:00
Ryuta Kamizono 7432e25187 Should respect attribute_types over column_types
Fixed the CI failure https://travis-ci.org/rails/rails/jobs/492291248#L1185-L1191.
2019-02-13 04:10:13 +09:00
Ryuta Kamizono 0ee96d13de Fix `pluck` and `select` with custom attributes
Currently custom attributes are always qualified by the table name in
the generated SQL wrongly even if the table doesn't have the named
column, it would cause an invalid SQL error.

Custom attributes should only be qualified if the table has the same
named column.
2019-02-13 02:47:46 +09:00
Ryuta Kamizono eb7c71bcd3 Avoid implicit delegation in the migration compatibility
The implicit delegation in the migration class is to be logged. It is
not intended in the migration compatibility, so it should be avoided.

Fixes #35224.
2019-02-12 07:18:40 +09:00
Gannon McGibbon 1bbf08bb49
Merge pull request #35196 from gmcgibbon/clarify_collection_proxy_docs
Clarify collection proxy docs
2019-02-11 10:29:46 -05:00
Ryuta Kamizono 93c2182836 Add `remove_foreign_key` for `change_table` 2019-02-11 23:46:07 +09:00
Ryuta Kamizono 713cee01a5 Fix typo a -> an, an -> a [ci skip] 2019-02-11 17:03:10 +09:00
Ryuta Kamizono da5843436b SQLite3: Implement `add_foreign_key` and `remove_foreign_key`
I implemented Foreign key create in `create_table` for SQLite3 at
#24743. This follows #24743 to implement `add_foreign_key` and
`remove_foreign_key`.
Unfortunately SQLite3 has one limitation that
`PRAGMA foreign_key_list(table-name)` doesn't have constraint name.
So we couldn't implement find/remove foreign key by name for now.

Fixes #35207.
Closes #31343.
2019-02-11 14:15:16 +09:00
Ryuta Kamizono d87afbf46f More exercise table name prefix and suffix tests 2019-02-11 13:55:06 +09:00
Ryuta Kamizono c83254db7a Merge pull request #35203 from chiastolite/add_column_without_column_names
Do not allow to add column without column name
2019-02-10 08:43:18 +09:00
Hiroyuki Morita 0914ea8477 Do not allow to add column without column name 2019-02-10 04:01:59 +09:00
Ryuta Kamizono c7721ea6b3 Refactor to extract defining column methods as `define_column_methods`
It makes to ease to handle all short-hand methods (e.g. validates
arguments etc).
2019-02-09 14:10:35 +09:00
Gannon McGibbon 7354d71f92 Clarify collection proxy docs
The docs for `ActiveRecord::Associations::CollectionProxy` describe
`ActiveRecord::Associations::Association`. I've moved them to
association and rewrote collection proxy's docs to be more applicable to
what the class actually does.`

[ci skip]
2019-02-08 11:11:19 -05:00
Ryuta Kamizono 5f9e05048d Refactor to just use `Association#target=` in `associate_records_to_owner`
`Association#target=` invokes `loaded!`, so we no longer need to call
the `loaded!` explicitly.

Since Preloader is private API, we don't guarantee that it behaves like
older version as long as using Preloader directly. But this refactoring
fortunately also fix the Preloader compatibility issue #35195.

Closes #35195.
2019-02-09 01:04:59 +09:00
Kasper Timm Hansen 216b8aa40f
Merge pull request #34618 from bogdanvlviv/fix-elapsed-time-calculations
Fix elapsed time calculations
2019-02-08 16:54:13 +01:00
Ryuta Kamizono 8cae8dcf54
Merge pull request #35193 from kamipo/fix_exists_with_distinct_and_offset
Fix `relation.exists?` with giving both `distinct` and `offset`
2019-02-08 23:48:21 +09:00
Ryuta Kamizono e133573c81
Merge pull request #35178 from bogdan/has-many-size
Bugfix has_many association #size when ids reader is cached and assoc…
2019-02-08 22:39:58 +09:00
bogdanvlviv dda9452314
Fix elapsed time calculations
I've found a few places in Rails code base where I think it makes sense
to calculate elapsed time more precisely by using
`Concurrent.monotonic_time`:

- Fix calculation of elapsed time in `ActiveSupport::Cache::MemoryStore#prune`
- Fix calculation of elapsed time in
  `ActiveRecord::ConnectionAdapters::ConnectionPool::Queue#wait_poll`
- Fix calculation of elapsed time in
  `ActiveRecord::ConnectionAdapters::ConnectionPool#attempt_to_checkout_all_existing_connections`
- Fix calculation of elapsed time in `ActiveRecord::ConnectionAdapters::Mysql2Adapter#explain`

See
https://docs.ruby-lang.org/en/2.5.0/Process.html#method-c-clock_gettime
https://blog.dnsimple.com/2018/03/elapsed-time-with-ruby-the-right-way

Related to 7c4542146f
2019-02-08 13:27:30 +00:00
Ryuta Kamizono 07dcd99a5a Fix `relation.exists?` with giving both `distinct` and `offset`
The `distinct` affects (reduces) rows of the result, so it is important
part when both `distinct` and `offset` are given.

Replacing SELECT clause to `1 AS one` and removing `distinct` and
`order` is just optimization for the `exists?`, we should not apply the
optimization for that case.

Fixes #35191.
2019-02-08 21:28:50 +09:00
Bogdan Gusiev 7f153e936a Bugfix has_many association #size when ids reader is cached and association is changed 2019-02-08 11:39:53 +02:00
Ryuta Kamizono 4cb1438b57 Refactor extracting `current_scope_restoring_block` into the scoping class
Relation is not best place to do this.
2019-02-08 11:12:08 +09:00
John Hawthorn 3302d2c2e6 Rename database_operations config to *_context 2019-02-07 15:41:43 -08:00
Eileen M. Uchitelle ac0dcbdc77
Merge pull request #35182 from jhawthorn/db-selection-refactor
Improve naming in DatabaseSelector
2019-02-07 18:27:26 -05:00
John Hawthorn a68bcde506 Rename database selector operations to context 2019-02-07 14:32:54 -08:00
Ryuta Kamizono 22360534ac Fix `relation.create` to avoid leaking scope to initialization block and callbacks
`relation.create` populates scope attributes to new record by `scoping`,
it is necessary to assign the scope attributes to the record and to find
STI subclass from the scope attributes.

But the effect of `scoping` is class global, it was caused undesired
behavior that pollute all class level querying methods in initialization
block and callbacks (`after_initialize`, `before_validation`,
`before_save`, etc), which are user provided code.

To avoid the leaking scope issue, restore the original current scope
before initialization block and callbacks are invoked.

Fixes #9894.
Fixes #17577.
Closes #31526.
2019-02-07 21:04:01 +09:00
Ryuta Kamizono 2e018361c7 Refactor around scoping
Don't use `false` as special value to skip to find inherited scope, we
could use `skip_inherited_scope = true`, and move `_scoping` back on
Relation.
2019-02-07 19:44:25 +09:00
John Hawthorn ed1f392ea6 Rename resolver ivar to operations in Resolver
We're already in the resolver, we call this class "operations" in the
middleware, so we should use the same naming here.
2019-02-06 11:57:30 -08:00
John Hawthorn 6aaf01385e Rename Session.build to Session.call
This is more consistent with Resolver, which has build called. This
allows using a Proc instead of a class, which could be nice if you need
to vary switching logic based on the request in a more ad-hoc way (ie.
check if it is an API request).
2019-02-06 11:55:53 -08:00
Yuya Tanaka 788eb51df3 Fix `CollectionProxy#concat` to return self by alias it to `#<<`
Formerly it was returning arguments (`records` array).
2019-02-06 15:01:48 +09:00
Aaron Patterson 9483cdee0a
Merge pull request #35171 from rails/speed-up-partials
Speed up partial rendering by caching "variable" calculation
2019-02-05 16:41:43 -08:00
Aaron Patterson 24b068bea1
Speed up partial rendering by caching "variable" calculation
This commit speeds up rendering partials by caching the variable name
calculation on the template.  The variable name is based on the "virtual
path" used for looking up the template.  The same virtual path
information lives on the template, so we can just ask the cached
template object for the variable.

This benchmark takes a couple files, so I'll cat them below:

```
[aaron@TC ~/g/r/actionview (speed-up-partials)]$ cat render_benchmark.rb
require "benchmark/ips"
require "action_view"
require "action_pack"
require "action_controller"

class TestController < ActionController::Base
end

TestController.view_paths = [File.expand_path("test/benchmarks")]
controller_view = TestController.new.view_context

result = Benchmark.ips do |x|
  x.report("render") do
    controller_view.render("many_partials")
  end
end
[aaron@TC ~/g/r/actionview (speed-up-partials)]$ cat test/benchmarks/test/_many_partials.html.erb
Looping:
<ul>
<% 100.times do |i| %>
  <%= render partial: "list_item", locals: { i: i } %>
<% end %>
</ul>
[aaron@TC ~/g/r/actionview (speed-up-partials)]$ cat test/benchmarks/test/_list_item.html.erb
<li>Number: <%= i %></li>
```

Benchmark results (master):

```
[aaron@TC ~/g/r/actionview (master)]$ be ruby render_benchmark.rb
Warming up --------------------------------------
              render    41.000  i/100ms
Calculating -------------------------------------
              render    424.269  (± 3.5%) i/s -      2.132k in   5.031455s
```

Benchmark results (this branch):

```
[aaron@TC ~/g/r/actionview (speed-up-partials)]$ be ruby render_benchmark.rb
Warming up --------------------------------------
              render    50.000  i/100ms
Calculating -------------------------------------
              render    521.862  (± 3.8%) i/s -      2.650k in   5.085885s
```
2019-02-05 15:24:21 -08:00
Ryuta Kamizono 0f3e8e1ebb Relation no longer respond to Arel methods
This follows up d97980a16d.
2019-02-06 04:58:45 +09:00
Ryuta Kamizono 2935d07569 Chaining named scope is no longer leaking to class level querying methods
Active Record uses `scoping` to delegate to named scopes from relations
for propagating the chaining source scope. It was needed to restore the
source scope in named scopes, but it was caused undesired behavior that
pollute all class level querying methods.

Example:

```ruby
class Topic < ActiveRecord::Base
  scope :toplevel, -> { where(parent_id: nil) }
  scope :children, -> { where.not(parent_id: nil) }
  scope :has_children, -> { where(id: Topic.children.select(:parent_id)) }
end

# Works as expected.
Topic.toplevel.where(id: Topic.children.select(:parent_id))

# Doesn't work due to leaking `toplevel` to `Topic.children`.
Topic.toplevel.has_children
```

Since #29301, the receiver in named scopes has changed from the model
class to the chaining source scope, so the polluting class level
querying methods is no longer required for that purpose.

Fixes #14003.
2019-02-06 00:37:08 +09:00
Ryuta Kamizono eec3e28a1a
Merge pull request #35127 from bogdan/counter-cache-loading
Bugfix association loading behavior when counter cache is zero
2019-02-05 20:32:37 +09:00
Bogdan Gusiev f1b64dff47 Bugfix association loading behavior when counter cache is zero 2019-02-05 11:58:59 +02:00
Aaron Patterson 3cace9eac9
Merge pull request #35154 from sponomarev/chore/sqlite1.4
Relax sqlite3 version dependency
2019-02-04 13:31:44 -08:00
Rafael Mendonça França cc2d614e63
Improve performance of blank? and present? in an ActiveRecord::Base instance
With this benchmark:

    require "bundler/setup"

    require "active_record"
    require "benchmark/ips"

    # This connection will do for database-independent bug reports.
    ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")

    ActiveRecord::Schema.define do
      create_table :posts, force: true do |t|
      end
    end

    class Post < ActiveRecord::Base
    end

    new_post = Post.new

    Benchmark.ips do |b|
      b.report("present?") do
        new_post.present?
      end

      b.report("blank?") do
        new_post.blank?
      end
    end

Before:

    Warming up --------------------------------------
                present?    52.147k i/100ms
                  blank?    53.077k i/100ms
    Calculating -------------------------------------
                present?    580.184k (±21.8%) i/s -      2.555M in   5.427085s
                  blank?    601.537k (± 9.2%) i/s -      2.972M in   5.003503s

After:

    Warming up --------------------------------------
                present?   378.235k i/100ms
                  blank?   375.476k i/100ms
    Calculating -------------------------------------
                present?     17.381M (± 7.5%) i/s -     86.238M in   5.001815s
                  blank?     17.877M (± 6.4%) i/s -     88.988M in   5.004634s

This improvement is mostly because those methods were hitting
`method_missing` on a lot of levels to be able to return the value.

To avoid all this stack walking we are short-circuiting those methods.

Closes #35059.
2019-02-04 15:56:47 -05:00
Sergey Ponomarev 2a29202288
Relax sqlite3 version dependency 2019-02-04 15:21:40 -05:00
Eileen M. Uchitelle 7d85d3847e
Merge pull request #35089 from eileencodes/fix-query-cache-for-database-switching
Invalidate all query caches for current thread
2019-02-04 08:46:10 -05:00
Eileen M. Uchitelle f4aed53e44
Merge pull request #35132 from eileencodes/allow-application-to-change-handler-names
Add ability to change the names of the default handlers
2019-02-04 08:45:48 -05:00
Gannon McGibbon 00f7dbab75
Merge pull request #35105 from olivierlacan/document-table-foreign-key
Hint at advanced options for foreign_key
2019-02-02 16:08:55 -05:00
Eileen Uchitelle 183c0eb472 Invalidate query cache for all connections in the current thread
This change ensures that all query cahces are cleared across all
connections per handler for the current thread so if you write on one
connection the read will have the query cache cleared.
2019-02-01 16:05:25 -05:00
Eileen Uchitelle 8d32346cde Add ability to change the names of the default handlers
When I wrote the `connected_to` and `connects_to` API's I wrote them
with the idea in mind that it didn't really matter what the
handlers/roles were called as long as those connecting to the roles knew
which one wrote and which one read.

With the introduction of the middleware Rails begins to assume it's
`writing` and `reading` and there's no room for other roles. At GitHub
we've been using this method for a long time so we have a ton of legacy
code that uses different handler names `default` and `readonly`. We
could rename all our code but I think this is better for a few reasons:

- Legacy apps that have been using multiple databases for a long time
  can have an eaiser time switching.
- If we later find this to cause more issues than it's worth we can
  easily deprecate.
- We won't force old apps to rewrite the resolver middleware just to use
  a different handler.

Adding the writing_role/reading_role required that I move the code that
creates the first handler for writing to the railtie. If I didn't move
this the core class would assign the handler before I was able to assign
a new one in my configuration and I'd end up with 3 handlers instead of
2.
2019-02-01 14:11:35 -05:00
Eileen M. Uchitelle caf8dbc159
Merge pull request #35130 from rails/move-delay-to-options-argument
Refactor options for database selector middleware
2019-02-01 13:53:34 -05:00
Rafael França bc0cad0690
Merge pull request #35082 from Shopify/eagerly-materialize-test-transactions
Eagerly materialize the fixtures transaction
2019-02-01 13:41:44 -05:00
Eileen Uchitelle 31f205234a Refactor options for middleware
Right now we only have one option that's supported, the delay. However I
can see us supporting other options in the future.

This PR refactors the options to get passed into the resolver so whether
you're using middleware or using the config options you can pass options
to the resolver. This will also make it easy to add new options in the
future.
2019-02-01 11:34:50 -05:00
Ryuta Kamizono 6127b8d920
Merge pull request #35116 from kamipo/fix_through_association_creation
Fix has_many through association creation
2019-02-01 09:13:43 +09:00
Ryuta Kamizono 0413501691 Remove unused attr_writers `visitor` and `indexes`
I deprecated two unused attr_writers `visitor` and `indexes` at 8056fe0
and f4bc364 conservatively, since those are accidentaly exposed in the
docs.

https://api.rubyonrails.org/v5.2/classes/ActiveRecord/ConnectionAdapters/AbstractAdapter.html
https://api.rubyonrails.org/v5.2/classes/ActiveRecord/ConnectionAdapters/TableDefinition.html

But I've found that `view_renderer` attr_writer is removed without
deprecation at #35093, that is also exposed in the doc.

https://api.rubyonrails.org/v5.2/classes/ActionView/Base.html

I'd like to also remove the deprecated attr_writers since I think that
removing `visitor` and `indexes` attr_writers is as safe as removing
`view_renderer` attr_writer.
2019-02-01 02:32:01 +09:00
Ryuta Kamizono 42d20e8de5 Revert "Merge pull request #33729 from kddeisz/plural-automatic-inverse"
This reverts commit ed1eda271c, reversing
changes made to 3d2caab7dc.

Reason: 7c3da6e003
2019-02-01 01:34:49 +09:00
alkesh26 379354572f ActiveRecord typo fixe. 2019-01-31 16:31:51 +05:30
Abhay Nikam 15db608f3e Fixed typo for DatabaseSelector::Resolver documentation 2019-01-31 07:57:38 +05:30
Ryuta Kamizono 1f5e21bcbc Remove redundant begin block
We enabled `Style/RedundantBegin` cop at #34764, but it is hard to
detect an offence if returning value put after the block.
2019-01-31 10:18:34 +09:00
Olivier Lacan 13f8beb435 Hint at advanced options for foreign_key
We sometimes display simple examples of additional parameters that can be
supplied to table-wise methods like these and I found it particularly difficult
to figure out which options `t.foreign_key` accepts without drilling very deep
into the specific SchemaStatements docs.

Since it's relatively common to create foreign keys with custom column names or
primary keys, it seems like this should help quite a few people.

[ci skip]
2019-01-30 19:29:03 -05:00
John Hawthorn ab952b1949 Write update_last_write_timestamp after request
We need to update using the timestamp from the end of the request, not
the start. For example, if a request spends 5+ seconds writing, we still
want to wait another 5 seconds for replication lag.

Since we now run the update after we yield, we need to use ensure to
make sure we update the timestamp even if there is an exception.
2019-01-30 13:25:09 -08:00
Eileen M. Uchitelle 8ca6bd2e94
Merge pull request #35073 from eileencodes/db-selection
Part 8: Multi db improvements, Adds basic automatic database switching to Rails
2019-01-30 14:03:45 -05:00
Eileen Uchitelle 0abcec416b Adds basic automatic database switching to Rails
The following PR adds behavior to Rails to allow an application to
automatically switch it's connection from the primary to the replica.

A request will be sent to the replica if:

* The request is a read request (`GET` or `HEAD`)
* AND It's been 2 seconds since the last write to the database (because
we don't want to send a user to a replica if the write hasn't made it
to the replica yet)

A request will be sent to the primary if:

* It's not a GET/HEAD request (ie is a POST, PATCH, etc)
* Has been less than 2 seconds since the last write to the database

The implementation that decides when to switch reads (the 2 seconds) is
"safe" to use in production but not recommended without adequate testing
with your infrastructure. At GitHub in addition to the a 5 second delay
we have a curcuit breaker that checks the replication delay
and will send the query to a replica before the 5 seconds has passed.
This is specific to our application and therefore not something Rails
should be doing for you. You'll need to test and implement more robust
handling of when to switch based on your infrastructure. The auto
switcher in Rails is meant to be a basic implementation / API that acts
as a guide for how to implement autoswitching.

The impementation here is meant to be strict enough that you know how to
implement your own resolver and operations classes but flexible enough
that we're not telling you how to do it.

The middleware is not included automatically and can be installed in
your application with the classes you want to use for the resolver and
operations passed in. If you don't pass any classes into the middleware
the Rails default Resolver and Session classes will be used.

The Resolver decides what parameters define when to
switch, Operations sets timestamps for the Resolver to read from. For
example you may want to use cookies instead of a session so you'd
implement a Resolver::Cookies class and pass that into the middleware
via configuration options.

```
config.active_record.database_selector = { delay: 2.seconds }
config.active_record.database_resolver = MyResolver
config.active_record.database_operations = MyResolver::MyCookies
```

Your classes can inherit from the existing classes and reimplment the
methods (or implement more methods) that you need to do the switching.
You only need to implement methods that you want to change. For example
if you wanted to set the session token for the last read from a replica
you would reimplement the `read_from_replica` method in your resolver
class and implement a method that updates a new timestamp in your
operations class.
2019-01-30 13:37:25 -05:00
Eileen Uchitelle dedcc19506 Fix case when we want a UrlConfig but the URL is nil
Previously if the `url` key in a config hash was nil we'd ignore the
configuration as invalid. This can happen when you're relying on a
`DATABASE_URL` in the env and that is not set in the environment.

```
production:
  <<: *default
  url: ENV['DATABASE_URL']
```

This PR fixes that case by checking if there is a `url` key in the
config instead of checking if the `url` is not nil in the config.

In addition to changing the conditional we then need to build a url hash
to merge with the original hash in the `UrlConfig` object.

Fixes #35091
2019-01-30 09:31:35 -05:00
Jean Boussier 74dbce0fca Eagerly materialize the fixtures transaction
The transaction used to restore fixtures is an implementation detail
that should be abstracted away. Idealy a test should behave the same
wether or not transactional fixtures are enabled.

However since transactions have been made lazy, the fixture
transaction started leaking into tests case. e.g. consider the
following (oversimplified) test:

```ruby
class SQLSubscriber
  attr_accessor :sql

  def initialize
    @sql = []
  end

  def call(*, event)
    sql << event[:sql]
  end
end

subscriber = SQLSubscriber.new
ActiveSupport::Notifications.subscribe("sql.active_record", subscriber)

User.connection.execute('SELECT 1', 'Generic name')
assert_equal ['SELECT 1'], subscriber.sql
```

On Rails 6 it starts to break because the `sql` array will be `['BEGIN', 'SELECT 1']`.

Several things are wrong here:

  - That transaction is not generated by the tested code, so it shouldn't be visible.
  - The transaction is not even closed yet, which again doesn't reflect the reality.
2019-01-29 16:10:47 +01:00
Ryuta Kamizono 8309cd2c68
Merge pull request #35071 from kamipo/text_without_limit
MySQL: Support `:size` option to change text and blob size
2019-01-29 17:02:04 +09:00
Aaron Patterson 5bb1ad59b1
Pull `@template` in to a local variable
This gets the PartialRenderer to be a bit closer to the
TemplateRenderer.  TemplateRenderer already keeps its template in a
local variable.
2019-01-28 14:55:12 -08:00
Aaron Patterson e98b51300a
Remove `@view` instance variable from the partial renderer
Similar to 1853b0d0ab
2019-01-28 14:22:32 -08:00
Ryuta Kamizono 1745e905a3 Allow changing text and blob size without giving the `limit` option
In MySQL, the text column size is 65,535 bytes by default (1 GiB in
PostgreSQL). It is sometimes too short when people want to use a text
column, so they sometimes change the text size to mediumtext (16 MiB) or
longtext (4 GiB) by giving the `limit` option.

Unlike MySQL, PostgreSQL doesn't allow the `limit` option for a text
column (raises ERROR: type modifier is not allowed for type "text").
So `limit: 4294967295` (longtext) couldn't be used in Action Text.

I've allowed changing text and blob size without giving the `limit`
option, it prevents that migration failure on PostgreSQL.
2019-01-29 06:49:32 +09:00
Lars Kanis c6d7e70be4 PostgreSQL: Use native timestamp decoders of pg-1.1
This improves performance of timestamp conversion and avoids
additional string allocations.
2019-01-26 20:30:43 +01:00
Ryuta Kamizono 57015cdfa2 Make `t.timestamps` with precision by default 2019-01-26 22:49:14 +09:00
Ryuta Kamizono 5fe6d3747e Fix `t.timestamps` missing `null: false` in `change_table bulk: true` 2019-01-26 21:19:20 +09:00
Ryuta Kamizono 17901d8764 Allow `column_exists?` giving options without type 2019-01-26 21:14:28 +09:00
Eileen M. Uchitelle bf3a8a0e6c
Merge pull request #35042 from eileencodes/fix-error-message-for-missing-handler
Fix error raised when handler doesn't exist
2019-01-25 10:56:00 -05:00
Eileen Uchitelle 1284f826cc Fix error raised when handler doesn't exist
While working on another feature for multiple databases (auto-switching)
I observed that in development the first request won't autoload the
application record connection for the primary database and may not yet
know about the replica connection.

In my test application this caused the application to thrown an error if
I tried to send the first request to the replica before the replica was
connected. This wouldn't be an issue in production because the
application is preloaded.

In order to fix this I decided to leave the original error message and
delete the new error message. I updated the original error message to
include the `role` to make it a bit clearer that the connection isn't
established for that particular role.

The error now reads:

```
No connection pool with 'primary' found for the 'reading' role.
```

A single database application will continue uisng the original error
message:

```
No connection pool with 'primary' found.
```
2019-01-25 09:56:43 -05:00
Kevin Deisz 5cf36e2ea2
Make `And` and `Case` into expression nodes
Allows aliasing, predications, ordering, and various other functions on `And` and `Case` nodes. This brings them in line with other nodes like `Binary` and `Unary`.
2019-01-24 14:35:52 -05:00
Ryuta Kamizono 1fecebae31 Allow `column_exists?` to be passed `type` argument as a string
Currently `conn.column_exists?("testings", "created_at", "datetime")`
returns false even if the table has the `created_at` column.

That reason is that `column.type` is a symbol but passed `type` is not
normalized to symbol unlike `column_name`, it is surprising behavior to
me.

I've improved that to normalize a value before comparison.
2019-01-24 19:02:44 +09:00
Xavier Noria 0065011660 Tell the user what to use instead of update_attributes/! 2019-01-23 22:26:22 +01:00
Dylan Thacker-Smith 5b58cc4f44 activerecord: Fix statement cache for strictly cast attributes 2019-01-23 14:49:42 -05:00
Yasuo Honda 3ea3cccc93 MySQL 8.0.14 adds `ER_FK_INCOMPATIBLE_COLUMNS`
https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-14.html
> Error messages relating to creating and dropping foreign keys
> were improved to be more specific and informative. (Bug #28526309, Bug #92087)

https://dev.mysql.com/doc/refman/8.0/en/server-error-reference.html

> Error number: 3780; Symbol: ER_FK_INCOMPATIBLE_COLUMNS; SQLSTATE: HY000
> Message: Referencing column '%s' and referenced column '%s' in foreign key constraint '%s' are incompatible.
> ER_FK_INCOMPATIBLE_COLUMNS was added in 8.0.14.
2019-01-22 15:15:52 +00:00
Kevin Deisz 65cec345b4
Alias case nodes
When `Arel` was merged into `ActiveRecord` we lost the ability to alias case nodes. This adds it back.
2019-01-21 10:28:39 -05:00
Ryuta Kamizono 4a0e3809be Fix type casting column default in `change_column`
Since #31230, `change_column` is executed as a bulk statement.

That caused incorrect type casting column default by looking up the
before changed type, not the after changed type.

In a bulk statement, we can't use `change_column_default_for_alter` if
the statement changes the column type.

This fixes the type casting to use the constructed target sql_type.

Fixes #34938.
2019-01-20 11:09:00 +09:00
Rafael Mendonça França 5a0230c67f
Preparing for 6.0.0.beta1 release 2019-01-18 15:42:12 -05:00
Dylan Thacker-Smith 592a961f29 activerecord: Fix where nil condition on composed_of attribute 2019-01-18 11:10:10 -05:00
Ryuta Kamizono ff3d1a42d5
Merge pull request #30000 from kamipo/all_of_queries_should_return_correct_result
All of queries should return correct result even if including large number
2019-01-18 22:58:54 +09:00
Eileen M. Uchitelle 7235e84935
Merge pull request #34969 from eileencodes/fix-error-message-for-multi-db-apps
Fix error message when adapter is not specified
2019-01-18 07:53:26 -05:00
Ryuta Kamizono c196ca72a0 Ensure `StatementCache#execute` never raises `RangeError`
Since 31ffbf8d, finder methods no longer raise `RangeError`. So
`StatementCache#execute` is the only place to raise the exception for
finder queries.

`StatementCache` is used for simple equality queries in the codebase.
This means that if `StatementCache#execute` raises `RangeError`, the
result could always be regarded as empty.
So `StatementCache#execute` just return nil in that range error case,
and treat that as empty in the caller side, then we can avoid catching
the exception in much places.
2019-01-18 16:01:14 +09:00
Ryuta Kamizono 31ffbf8d50 All of queries should return correct result even if including large number
Currently several queries cannot return correct result due to incorrect
`RangeError` handling.

First example:

```ruby
assert_equal true, Topic.where(id: [1, 9223372036854775808]).exists?
assert_equal true, Topic.where.not(id: 9223372036854775808).exists?
```

The first example is obviously to be true, but currently it returns
false.

Second example:

```ruby
assert_equal topics(:first), Topic.where(id: 1..9223372036854775808).find(1)
```

The second example also should return the object, but currently it
raises `RecordNotFound`.

It can be seen from the examples, the queries including large number
assuming empty result is not always correct.

Therefore, This change handles `RangeError` to generate executable SQL
instead of raising `RangeError` to users to always return correct
result. By this change, it is no longer raised `RangeError` to users.
2019-01-18 16:01:07 +09:00
Ryuta Kamizono 5b6daff5b6 Use `unboundable?` rather than `boundable?`
The `unboundable?` behaves like the `infinite?`.

```ruby
inf = Topic.predicate_builder.build_bind_attribute(:id, Float::INFINITY)
inf.infinite?    # => 1

oob = Topic.predicate_builder.build_bind_attribute(:id, 9999999999999999999999999999999)
oob.unboundable? # => 1

inf = Topic.predicate_builder.build_bind_attribute(:id, -Float::INFINITY)
inf.infinite?    # => -1

oob = Topic.predicate_builder.build_bind_attribute(:id, -9999999999999999999999999999999)
oob.unboundable? # => -1
```
2019-01-18 14:56:43 +09:00
Rafael França 41dc4d9490
Merge pull request #34963 from dylanahsmith/better-composed-of-single-field-query
activerecord: Use a simpler query condition for aggregates with one mapping
2019-01-17 18:32:27 -05:00
Eileen Uchitelle 83b995206a Fix error message when adapter is not specified
When we added support for multiple databases through a 3-tiered config
and configuration objects this error message got a bit convoluted.

Previously if you had an application with a missing configuation and
multiple databases the error message would look like this:

```
'doesnexist' database is not configured. Available: development,
development, test, test, production, production
(ActiveRecord::AdapterNotSpecified)
```

That's not very descriptive since it duplicates the environments
(because there are multiple databases per environment for this
application).

To fix this I've constructed a bit more readable error message which now
reads like this if you have a multi db app:

```
The `doesntexist` database is not configured for the `production`
environment. (ActiveRecord::AdapterNotSpecified)

Available databases configurations are:

development: primary, primary_readonly
test: primary, primary_readonly
production: primary, primary_readonly
```

And like this if you have a single db app:

```
The `doesntexist` database is not configured for the `production`
environment. (ActiveRecord::AdapterNotSpecified)

Available databases configurations are:

development
test
```

This makes the error message more readable and presents the user all
available options for the database connections.
2019-01-17 17:26:32 -05:00
Ryuta Kamizono 9905cdc946
Use public_send instead since respond_to? doesn't include private/protected methods by default
Co-Authored-By: dylanahsmith <dylan.smith@shopify.com>
2019-01-17 17:00:10 -05:00
Rafael Mendonça França 6c745b0c51
Remove deprecated `#set_state` from the transaction object 2019-01-17 16:08:33 -05:00
Rafael Mendonça França 5f3ed87843
Remove deprecated `#supports_statement_cache?` from the database adapters 2019-01-17 16:08:33 -05:00
Rafael Mendonça França 400ba786e1
Remove deprecated `#insert_fixtures` from the database adapters 2019-01-17 16:08:33 -05:00
Rafael Mendonça França 45b4d5f81f
Remove deprecated `ActiveRecord::ConnectionAdapters::SQLite3Adapter#valid_alter_table_type?` 2019-01-17 16:08:33 -05:00
Rafael Mendonça França 91ddb30083
Do not allow passing the column name to `sum` when a block is passed 2019-01-17 16:08:33 -05:00
Rafael Mendonça França 67356f2034
Do not allow passing the column name to `count` when a block is passed 2019-01-17 16:08:33 -05:00
Rafael Mendonça França d97980a16d
Remove delegation of missing methods in a relation to arel 2019-01-17 16:08:32 -05:00
Rafael Mendonça França a7becf147a
Remove delegation of missing methods in a relation to private methods of the class 2019-01-17 16:08:32 -05:00
Rafael Mendonça França f59b08119b
Change `SQLite3Adapter` to always represent boolean values as integers 2019-01-17 16:08:32 -05:00
Rafael Mendonça França 0bef23e630
Remove ability to specify a timestamp name for `#cache_key` 2019-01-17 16:08:32 -05:00
Rafael Mendonça França 90d7842186
Remove deprecated `ActiveRecord::Migrator.migrations_path=` 2019-01-17 16:08:32 -05:00
Rafael Mendonça França 27b252d6a8
Remove deprecated `expand_hash_conditions_for_aggregates` 2019-01-17 16:08:32 -05:00
Dylan Thacker-Smith b278db1d23 Avoid using yield_self to make it easier to backport 2019-01-17 14:16:13 -05:00
Dylan Thacker-Smith 242fc54f57 activerecord: Use a simpler query condition for aggregates with one mapping 2019-01-17 13:37:11 -05:00
bogdanvlviv 2bad3f46cd
Add foreign key to active_storage_attachments for `blob_id` via new migration
We need this in order to be able to add this migration for users that
use ActiveStorage during update their apps from Rails 5.2 to Rails 6.0.

Related to #33405

`rake app:update` should update active_storage

`rake app:update` should execute `rake active_storage:update`
if it is used in the app that is being updated.
It will add new active_storage's migrations to users' apps during update Rails.

Context https://github.com/rails/rails/pull/33405#discussion_r204239399

Also, see a related discussion in the Campfire:
https://3.basecamp.com/3076981/buckets/24956/chats/12416418@1236713081
2019-01-16 13:13:23 +00:00
Laerti 41ffddbc8b Refs #28025 nullify *_type column on polymorphic associations on :nu… (#28078)
This PR addresses the issue described in #28025. On `dependent: :nullify` strategy only the foreign key of the relation is nullified. However on polymorphic associations the  `*_type` column is not nullified leaving the record with a NULL `*_id` but the `*_type` column is present.
2019-01-15 23:03:20 +09:00
Ryuta Kamizono e4213e7c68 Remove public `prevent_writes` writer
The `@prevent_writes` should be updated only in the
`while_preventing_writes`, it is not necessary to expose the attr
writer.
2019-01-15 22:55:31 +09:00
Ryuta Kamizono 8056fe0d96 Deprecate `connection.visitor = ...` which is not released internal usage
This attr writer was introduced at 7db90aa, but the usage is already
removed at bd2f5c0 before v3.2.0.rc1 is released.

If we'd like to customize the visitor in the connection, `arel_visitor`
which is implemented in all adapters (mysql2, postgresql, sqlite3,
oracle-enhanced, sqlserver) could be used for the purpose #23515.
2019-01-15 22:44:27 +09:00
Ryuta Kamizono 862c78b26f Remove unused `Arel::Compatibility::Wheres`
This class is no longer used since 9cbfc8a370.
2019-01-15 22:13:25 +09:00
Rafael França fcd38cf19f
Merge pull request #34891 from gmcgibbon/ac_params_exists
Allow strong params in ActiveRecord::Base#exists?
2019-01-14 17:34:44 -05:00
Ryuta Kamizono bb75d68fe2 More exercise test cases for `not_between`
And support endless ranges for `not_between` like as `between`.

Follow up #34906.
2019-01-12 07:44:27 +09:00
Aaron Patterson 74bef0970f
Merge branch 'master' into ac_params_exists 2019-01-11 11:21:53 -08:00
Aaron Patterson 3a1b2a2196
Merge pull request #34906 from gregnavis/add-endless-ranges-to-activerecord
Support endless ranges in where
2019-01-11 11:20:05 -08:00
Greg Navis 7110dbea00 Support endless ranges in where
This commit adds support for endless ranges, e.g. (1..), that were added
in Ruby 2.6. They're functionally equivalent to explicitly specifying
Float::INFINITY as the end of the range.
2019-01-11 14:27:34 +01:00
Ryuta Kamizono 5f9e0f848e Remove `id_value` argument which is no longer passed to `sql_for_insert`
Since #26002, `id_value` is no longer passed to `sql_for_insert`.
2019-01-11 19:55:14 +09:00
Ryuta Kamizono 6c6c32463e Refactor `bind_attribute` to expand an association to actual attribute 2019-01-11 18:01:55 +09:00
Ryuta Kamizono eb5fef554f Refactor `build_relation` in the uniqueness validator to avoid low level predicate construction 2019-01-11 17:32:48 +09:00
Ryuta Kamizono ea65d92f19
Enable `Lint/UselessAssignment` cop to avoid unused variable warnings (#34904)
* Enable `Lint/UselessAssignment` cop to avoid unused variable warnings

Since we've addressed the warning "assigned but unused variable"
frequently.

370537de05
3040446cec
5ed618e192
76ebafe594

And also, I've found the unused args in c1b14ad which raises no warnings
by the cop, it shows the value of the cop.
2019-01-09 18:09:01 +09:00
Ryuta Kamizono 7f856c3c8d Consolidate the duplicated code that building range predicate
This slightly change the code in the Arel to allow +/-INFINITY as open
ended since the Active Record expects that behavior. See 5ecbeda.
2019-01-08 16:45:17 +09:00
bannzai 45cfe9f8b6 ♻️ Fix mysql type map for enum and set 2019-01-08 14:56:46 +09:00
Gannon McGibbon 1e923b4984 Allow strong params in ActiveRecord::Base#exists?
Allow `ActionController::Params` as argument of
`ActiveRecord::Base#exists?`
2019-01-07 14:59:02 -05:00
Eileen M. Uchitelle 725c6422e7
Merge pull request #34773 from eileencodes/share-fixture-connections-with-multiple-handlers
For fixtures share the connection pool when there are multiple handlers
2019-01-04 08:49:17 -05:00
Ryuta Kamizono a5a22c4ea1
Merge pull request #34858 from albertoalmagro/make-rails-compatible-accross-ruby-versions
Make average compatible across ruby versions
2019-01-04 10:32:37 +09:00
Alberto Almagro d237c7c72c Make average compatible accross Ruby versions
Since Ruby 2.6.0 NilClass#to_d is returning `BigDecimal` 0.0, this
breaks `average` compatibility with prior Ruby versions. This patch
makes `average` return `nil` in all Ruby versions when there are no
rows.
2019-01-04 00:41:22 +01:00
Ryuta Kamizono 7a4e20e0cd Merge the redundant `when Symbol` case to the `when String, ...` 2019-01-04 08:05:17 +09:00
Ryuta Kamizono e9aa0c5c72 2x faster `connection.type_cast`
`nil`, `Numeric`, and `String` are most basic objects which are passed
to `type_cast`. But now each `when *types_which_need_no_typecasting`
evaluation allocates extra two arrays, it makes `type_cast` slower.

The `types_which_need_no_typecasting` was introduced at #15351, but the
method isn't useful (never used any adapters) since all adapters
(sqlite3, mysql2, postgresql, oracle-enhanced, sqlserver) still
overrides the `_type_cast`.

Just expanding the method would make the `type_cast` 2x faster.

```ruby
module ActiveRecord
  module TypeCastFast
    def type_cast_fast(value, column = nil)
      value = id_value_for_database(value) if value.is_a?(Base)

      if column
        value = type_cast_from_column(column, value)
      end

      _type_cast_fast(value)
    rescue TypeError
      to_type = column ? " to #{column.type}" : ""
      raise TypeError, "can't cast #{value.class}#{to_type}"
    end

    private
      def _type_cast_fast(value)
        case value
        when Symbol, ActiveSupport::Multibyte::Chars, Type::Binary::Data
          value.to_s
        when true       then unquoted_true
        when false      then unquoted_false
        # BigDecimals need to be put in a non-normalized form and quoted.
        when BigDecimal then value.to_s("F")
        when nil, Numeric, String then value
        when Type::Time::Value then quoted_time(value)
        when Date, Time then quoted_date(value)
        else raise TypeError
        end
      end
  end
end

conn = ActiveRecord::Base.connection
conn.extend ActiveRecord::TypeCastFast

Benchmark.ips do |x|
  x.report("type_cast") { conn.type_cast("foo") }
  x.report("type_cast_fast") { conn.type_cast_fast("foo") }
  x.compare!
end
```

```
Warming up --------------------------------------
           type_cast    58.733k i/100ms
      type_cast_fast   101.364k i/100ms
Calculating -------------------------------------
           type_cast    708.066k (± 5.9%) i/s -      3.583M in   5.080866s
      type_cast_fast      1.424M (± 2.3%) i/s -      7.197M in   5.055860s

Comparison:
      type_cast_fast:  1424240.0 i/s
           type_cast:   708066.0 i/s - 2.01x  slower
```
2019-01-04 07:24:57 +09:00
Eileen Uchitelle b24bfcce77 Share the connection pool when there are multiple handlers
In an application that has a primary and replica database the data
inserted on the primary connection will not be able to be read by the
replica connection.

In a test like this:

```
test "creating a home and then reading it" do
  home = Home.create!(owner: "eileencodes")

  ActiveRecord::Base.connected_to(role: :default) do
    assert_equal 3, Home.count
  end

  ActiveRecord::Base.connected_to(role: :readonly) do
    assert_equal 3, Home.count
  end
end
```

The home inserted in the beginning of the test can't be read by the
replica database because when the test is started a transasction is
opened byy `setup_fixtures`. That transaction remains open for the
remainder of the test until we are done and run `teardown_fixtures`.

Because the data isn't actually committed to the database the replica
database cannot see the data insertion.

I considered a couple ways to fix this. I could have written a database
cleaner like class that would allow the data to be committed and then
clean up that data afterwards. But database cleaners can make the
database slow and the point of the fixtures is to be fast.

In GitHub we solve this by sharing the connection pool for the replicas
with the primary (writing) connection. This is a bit hacky but it works.
Additionally since we define `replica? || preventing_writes?` as the
code that blocks writes to the database this will still prevent writing
on the replica / readonly connection. So we get all the behavior of
multiple connections for the same database without slowing down the
database.

In this PR the code loops through the handlers. If the handler doesn't
match the default handler then it retrieves the connection pool from the
default / writing handler and assigns the reading handler's connections
to that pool.

Then in enlist_fixture_connections it maps all the connections for the
default handler because all the connections are now available on that
handler so we don't need to loop through them again.

The test uses a temporary connection pool so we can test this with
sqlite3_mem. This adapter doesn't behave the same as the others and
after looking over how the query cache test works I think this is the
most correct. The issues comes when calling `connects_to` because that
establishes new connections and confuses the sqlite3_mem adapter. I'm
not entirely sure why but I wanted to make sure we tested all adapters
for this change and I checked that it wasn't the shared connection code
that was causing issues - it was the `connects_to` code.
2019-01-03 16:32:46 -05:00
Kasper Timm Hansen 6f0cda8f8e
Merge pull request #33985 from eugeneius/attribute_methods_schema_cache
Only define attribute methods from schema cache
2019-01-03 15:03:06 +01:00
Ryuta Kamizono 260b273106
Merge pull request #34836 from kamipo/class_level_update_without_ids
Restore an ability that class level `update` without giving ids
2019-01-02 23:42:16 +09:00
Ryuta Kamizono fb6743acc5 MariaDB: Remove version checking lower the 5.5.8
Since we already bumped the minimum version of MySQL to 5.5.8 at #33853.
2019-01-02 10:50:21 +09:00
Ryuta Kamizono 3d5db4920a Add test case for `preventing_writes?`
Since the `preventing_writes?` is public API.
2019-01-02 08:42:50 +09:00
yuuji.yaginuma 2484443aa8 Bring ActiveRecord::Core's API document back [ci skip]
If exist `:nodoc:` before method define, it affects all subsequent
method definitions.
2019-01-02 08:32:17 +09:00
Ryuta Kamizono 4c05434973 Restore an ability that class level `update` without giving ids
That ability was introduced at #11898 as `Relation#update` without
giving ids, so the ability on the class level is not documented and not
tested.

c83e30d which fixes #33470 has lost two undocumented abilities.

One has fixed at 5c65688, but I missed the ability on the class level.

Removing any feature should not be suddenly happened in a stable version
even if that is not documented.

I've restored the ability and added test case to avoid any regression in
the future.

Fixes #34743.
2019-01-02 06:23:52 +09:00
Ryuta Kamizono 600a66f60c Fix TypeError: no implicit conversion of Arel::Attributes::Attribute into String properly
This reverts 27c6c07 since `arel_attr.to_s` is not right way to avoid
the type error.

That to_s returns `"#<struct Arel::Attributes::Attribute ...>"`, there
is no reason to match the regex to the inspect form.

And also, the regex path is not covered by our test cases. I've tweaked
the regex for redundant part and added assertions for the regex path.
2019-01-02 04:15:32 +09:00
Arun Agrawal 50e3680768 Bump license years for 2019 2018-12-31 10:24:38 +07:00
Ryuta Kamizono 22a6ff68b1 Use high level API on `migration_context` instead of using low level API directly
Since `migration_context` has `migrations_paths` itself and provides
methods which returning values from parsed migration files, so there is
no reason to use the `parse_migration_filename` low level API directly.
2018-12-28 10:17:12 +09:00
Ryuta Kamizono c1b14aded2 Deprecate passing `migrations_paths` to `connection.assume_migrated_upto_version`
Since #31727, `migrations_paths` in `assume_migrated_upto_version` is no
longer used.
2018-12-28 09:21:09 +09:00
Ryuta Kamizono 4ae8d6182f Merge pull request #34806 from bogdan/reuse-find-target
Reuse AR::Association#find_target method
2018-12-27 19:40:10 +09:00
Bogdan Gusiev bc30d421f3 Reuse AR::Association#find_target method 2018-12-27 11:53:27 +02:00
Eileen M. Uchitelle b00e46bd6d
Merge pull request #34753 from eileencodes/raise-less-confusing-error-if-handler-doesnt-exist
Raise helpful error when role doesn't exist
2018-12-21 11:43:35 -05:00
Eileen Uchitelle 22a1265828 Raise helpful error when role doesn't exist
If you try to call `connected_to` with a role that doesn't have an
established connection you used to get an error that said:

```
>> ActiveRecord::Base.connected_to(role: :i_dont_exist) { Home.first }

ActiveRecord::ConnectionNotEstablished Exception: No connection pool
with 'primary' found.
```

This is confusing because the connection could be established but we
spelled the role wrong.

I've changed this to raise if the `role` used in `connected_to` doesn't
have an associated handler. Users who encounter this should either check
that the role is spelled correctly (writin -> writing), establish a
connection to that role in the model with connects_to, or use the
`database` keyword for the `role`.

I think this will provide a less confusing error message for those
starting out with multiple databases.
2018-12-21 11:10:10 -05:00
Ryuta Kamizono 721e26767c
Merge pull request #34742 from kamipo/row_format_dynamic_by_default
MySQL: `ROW_FORMAT=DYNAMIC` create table option by default
2018-12-21 17:39:31 +09:00
Ryuta Kamizono 892e38c78e Enable `Style/RedundantBegin` cop to avoid newly adding redundant begin block
Currently we sometimes find a redundant begin block in code review
(e.g. https://github.com/rails/rails/pull/33604#discussion_r209784205).

I'd like to enable `Style/RedundantBegin` cop to avoid that, since
rescue/else/ensure are allowed inside do/end blocks in Ruby 2.5
(https://bugs.ruby-lang.org/issues/12906), so we'd probably meets with
that situation than before.
2018-12-21 06:12:42 +09:00
Ryuta Kamizono 8034dde023 Module#{define_method,alias_method,undef_method,remove_method} become public since Ruby 2.5
https://bugs.ruby-lang.org/issues/14133
2018-12-21 01:39:18 +09:00
Ryuta Kamizono 3e50a1bcd4
Merge pull request #30973 from k0kubun/prefer-block-parameter
Unify _read_attribute definition to use &block
2018-12-20 17:38:32 +09:00
Takashi Kokubun f3c866a743 Unify _read_attribute definition to use &block
Thanks to ko1, passing block parameter to another method is
significantly optimized in Ruby 2.5.
https://bugs.ruby-lang.org/issues/14045

Thus we no longer need to keep this ugly hack.
2018-12-20 10:37:15 +09:00
Ryuta Kamizono a1652c196e MySQL: `ROW_FORMAT=DYNAMIC` create table option by default
Since MySQL 5.7.9, the `innodb_default_row_format` option defines the
default row format for InnoDB tables. The default setting is `DYNAMIC`.

The row format is required for indexing on `varchar(255)` with `utf8mb4`
columns.

As long as using MySQL 5.6, CI won't be passed even if MySQL server
setting is properly configured the same as MySQL 5.7
(`innodb_file_per_table = 1`, `innodb_file_format = 'Barracuda'`, and
`innodb_large_prefix = 1`) since InnoDB table is created as the row
format `COMPACT` by default on MySQL 5.6, therefore indexing on string
with `utf8mb4` columns aren't succeeded.

Making `ROW_FORMAT=DYNAMIC` create table option by default for legacy
MySQL version would mitigate the indexing issue on the user side, and it
makes CI would be passed on MySQL 5.6 which is configured properly.
2018-12-19 08:53:16 +09:00
Ryuta Kamizono 0fa5b5510c Use `utf8mb4` charset for internal tables if the row format `DYNAMIC` by default
The indexing issue on `utf8mb4` columns is resolved since MySQL 5.7.9.
2018-12-19 08:38:12 +09:00
Ryuta Kamizono 7d83ef4f7b Ensure that preventing writes is invoked before `materialize_transactions` consistently 2018-12-12 23:14:26 +09:00
John Hawthorn b67a3f5cac Add AR::Base.connected_to?
This can be used to check the currently connected role. It's meant to
mirror AR::Base.connected_to
2018-12-11 14:24:02 -08:00
Ryuta Kamizono 5742344024 An empty transaction does not raise the `ReadOnlyError` if preventing writes
BEGIN transaction would cause COMMIT or ROLLBACK, so unless COMMIT and
ROLLBACK aren't treated as write queries as well as BEGIN, the
`ReadOnlyError` would be raised.
2018-12-11 21:07:37 +09:00
Rafael França b310827506
Merge pull request #34666 from vinistock/upgrade_rubocop_and_fix_offenses
Upgrade Rubocop to 0.61.1 and fix offenses
2018-12-10 17:38:47 -05:00
Ryuta Kamizono 10457475b9 An explain query does not raise the `ReadOnlyError` if preventing writes 2018-12-11 07:23:00 +09:00
Ryuta Kamizono 07f0f1a8c7 Don't treat begin and rollback transactions as write queries
Otherwise `save` method would raise the `ReadOnlyError` against `BEGIN`
and `ROLLBACK` queries.
2018-12-11 06:40:38 +09:00
Vinicius Stock 3b7a4d3d75
Upgrade Rubocop to 0.61.1 and fix offenses 2018-12-10 19:22:56 -02:00
Ryuta Kamizono cf71f31e2e Prevent write queries with prepared statements for mysql2 adapter
Without this change, mysql2 adapter with prepared statements won't pass
`base_test.rb`.

```
% ARCONN=mysql2 be ruby -w -Itest test/cases/base_test.rb
Using mysql2
Run options: --seed 27614

# Running:

....S..............................F

Failure:
BasicsTest#test_creating_a_record_raises_if_preventing_writes [test/cases/base_test.rb:1493]:
ActiveRecord::ReadOnlyError expected but nothing was raised.

rails test test/cases/base_test.rb:1492

...F

Failure:
BasicsTest#test_deleting_a_record_raises_if_preventing_writes [test/cases/base_test.rb:1513]:
ActiveRecord::ReadOnlyError expected but nothing was raised.

rails test test/cases/base_test.rb:1510

............................................................................................................F

Failure:
BasicsTest#test_updating_a_record_raises_if_preventing_writes [test/cases/base_test.rb:1503]:
ActiveRecord::ReadOnlyError expected but nothing was raised.

rails test test/cases/base_test.rb:1500

..........

Finished in 2.534490s, 62.7345 runs/s, 149.5370 assertions/s.
159 runs, 379 assertions, 3 failures, 0 errors, 1 skips
```
2018-12-11 05:08:59 +09:00
Ryuta Kamizono bde898c1df Prevent write queries for `exec_query`
Follow up #34505.
2018-12-11 04:51:54 +09:00
Eileen M. Uchitelle bdc96802a7
Merge pull request #34632 from rails/rename-write-on-read-error
Rename error that occurs when writing on a read
2018-12-07 17:54:33 -05:00
Bogdan 18f83faf71 #create_or_find_by/!: add more tests and fix docs (#34653)
* `#create_or_find_by/!`: add more tests

* Fix docs of `create_or_find_by`

This method uses `find_by!` internally.
2018-12-08 07:46:30 +09:00
Eileen Uchitelle db54afba2e Rename error that occurs when writing on a read
I originally named this `StatementInvalid` because that's what we do in
GitHub, but `@tenderlove` pointed out that this means apps can't test
for or explitly rescue this error. `StatementInvalid` is pretty broad so
I've renamed this to `ReadOnlyError`.
2018-12-07 08:53:47 -05:00
Gannon McGibbon 93f19071ad Fix join table column quoting with SQLite. 2018-12-05 11:56:14 -05:00
Alfred Dominic c4d157aafe option to disable scopes that `ActiveRecord.enum` generates by default 2018-12-04 15:56:31 +05:30
Ryuta Kamizono 609c58bfa6 Merge pull request #34609 from kamipo/delete_all_on_collection_proxy
Ensure that `delete_all` on collection proxy returns affected count
2018-12-04 15:48:22 +09:00
Ryuta Kamizono 4d7354aa35 Ensure that `delete_all` on collection proxy returns affected count
Unlike the `Relation#delete_all`, `delete_all` on collection proxy
doesn't return affected count. Since the `CollectionProxy` is a subclass
of the `Relation`, this inconsistency is probably not intended, so it
should return the count consistently.
2018-12-04 14:33:31 +09:00
Gannon McGibbon 67bca35a64 Reset scope after collection delete
Reset scope after delete on collection association to clear stale
offsets of removed records.
2018-12-04 00:16:42 -05:00
Rafael França e167818687
Merge pull request #34602 from guizmaii/master
Pass the `connection` to the `@instrumenter.instrument` method call
2018-12-03 10:47:50 -05:00
Ryuta Kamizono df76eaa4f1 Address "warning: shadowing outer local variable - parts"
And hide the `READ_QUERY` internal constant.
2018-12-03 19:57:27 +09:00
jules Ivanic 6d1c9a9b23 Pass the `connection` to the `@instrumenter.instrument` method call 2018-12-03 10:12:14 +01:00
Gannon McGibbon 220494185c Clarify no support for non PK id columns
[ci skip]
2018-11-30 13:25:42 -05:00
Eileen Uchitelle f39d72d526 Add ability to prevent writes to a database
This PR adds the ability to prevent writes to a database even if the
database user is able to write (ie the database is a primary and not a
replica).

This is useful for a few reasons: 1) when converting your database from
a single db to a primary/replica setup - you can fix all the writes on
reads early on, 2) when we implement automatic database switching or
when an app is manually switching connections this feature can be used
to ensure reads are reading and writes are writing. We want to make sure
we raise if we ever try to write in read mode, regardless of database
type and 3) for local development if you don't want to set up multiple
databases but do want to support rw/ro queries.

This should be used in conjunction with `connected_to` in write mode.
For example:

```
ActiveRecord::Base.connected_to(role: :writing) do
  Dog.connection.while_preventing_writes do
    Dog.create! # will raise because we're preventing writes
  end
end

ActiveRecord::Base.connected_to(role: :reading) do
  Dog.connection.while_preventing_writes do
    Dog.first # will not raise because we're not writing
  end
end
```
2018-11-30 09:28:04 -05:00
Ryuta Kamizono 5c6316dbb8
Merge pull request #34572 from kamipo/fix_scoping_with_query_method
Fix the scoping with query methods in the scope block
2018-11-30 21:41:12 +09:00
Ryuta Kamizono faf91b0db4 Use the full link URL instead of bit.ly [ci skip] 2018-11-30 21:37:28 +09:00
Ryuta Kamizono 3090b35848 Fix the scoping with query methods in the scope block
Follow up #33394.

#33394 only fixes the case of scoping with klass methods in the scope
block which invokes `klass.all`.
Query methods in the scope block also need to invoke `klass.all` to be
affected by the scoping.
2018-11-30 05:50:40 +09:00
Gannon McGibbon 72e63c71bb Allow aliased attributes in update
Allow aliased attributes to be used in `#update_columns` and `#update`.
2018-11-29 14:49:17 -05:00
Rafael França 89595aff8e
Merge pull request #34562 from ruralocity/active-record-query-docs-improvement
Improve ActiveRecord::Querying documentation [ci skip]
2018-11-28 18:07:24 -05:00
Aaron Sumner 96f6a59d8e Improve ActiveRecord::Querying documentation [ci skip]
* Break up long sentences
* Reword some sentences to clarify subject, predicate, and object
* Explain drawbacks of using count_by_sql
2018-11-28 14:50:25 -08:00
Gannon McGibbon 03fadebe0e Allow spaces in postgres table names
Fixes issue where "user post" is misinterpreted as "\"user\".\"post\""
when quoting table names with the postgres adapter.
2018-11-28 16:20:22 -05:00
Rafael França 9c6e362a63
Merge pull request #34557 from sergioisidoro/sergio-patch-load-error
Patch load error in case GemSpecError
2018-11-28 12:51:39 -05:00
sergioisidoro a1a08ac7ca
Patch load error in case GemSpecError 2018-11-28 16:57:32 +02:00
Sean Griffin 9c4d3c341c
Merge pull request #33835 from schneems/schneems/faster_cache_version
Use raw time string from DB to generate ActiveRecord#cache_version
2018-11-27 18:51:20 -07:00
Rafael França ddaca7ccec
Merge pull request #34528 from DmitryTsepelev/fix-ignored-attributes
Additional types of ResultSet should not contain keys of #attributes_to_define_after_schema_loads
2018-11-27 14:43:01 -05:00
Ryuta Kamizono 93c94973ab Revert "Merge pull request #34538 from bogdan/reuse-find-target"
This reverts commit f2ab8b64d4, reversing
changes made to b9c7305dbe.

Reason: `scope.take` is not the same with `scope.to_a.first`.
2018-11-28 01:09:35 +09:00
Eileen M. Uchitelle f2ab8b64d4
Merge pull request #34538 from bogdan/reuse-find-target
Reuse code in AR::Association#find_target
2018-11-27 10:58:10 -05:00
DmitryTsepelev d34c1fc3d6 Cached columns_hash fields should be excluded from ResultSet#column_types 2018-11-27 16:30:26 +03:00
Eileen M. Uchitelle b9c7305dbe
Merge pull request #34480 from tekin/configurable-implicit-ordering-column
Make it possible to override the implicit order column
2018-11-27 08:19:50 -05:00