Commit Graph

267 Commits

Author SHA1 Message Date
Jean Boussier 60df08920f
Merge pull request #50373 from fatkodima/improve-queries-assertions-matchers
Expose `assert_queries_match` and `assert_no_queries_match` assertions
2023-12-21 12:59:10 +01:00
fatkodima f48bbff32c Expose `assert_queries_match` and `assert_no_queries_match` assertions 2023-12-21 01:30:16 +02:00
fatkodima f68727dcd0 Fix async queries to work with query cache 2023-12-20 23:50:41 +02:00
Jean Boussier f8e842f13e Fix `Model.none.async_pluck` to return a Promise
Previously it would return a naked empty array, which is a break
of the interface.
2023-12-14 12:36:42 +01:00
Akhil G Krishnan 80a5fb50c2 Update postgres naming to PostgreSQL 2023-10-10 13:44:20 +05:30
Shouichi Kamiya 51ac8b9f6f Enable Minitest/LiteralAsActualArgument
There are assertions that expected/actual arguments are passed in the
reversed order by mistake. Enabling the LiteralAsActualArgument rule
prevents this mistake from happening.

The existing tests were auto-corrected by rubocop with a bit of
indentation adjustment.

Co-authored-by: Jonathan Hefner <jonathan@hefner.pro>
2023-09-13 10:09:32 +09:00
Rafael Mendonça França 9eb16e77da
Revert "Merge pull request #49242 from eileencodes/fix-inferred-query-constraints"
This reverts commit 213e8ff95e, reversing
changes made to b5d63b9b16.

It broke a lot of test.

It also changed how lazy loading of model code works.

With this change a simple `belongs_to :owner` is now loading `owner.rb`
when that line is executed, where before it wasn't doing that.
This can have serious impact on boot time in development.

We should make this as lazy as possible.
2023-09-12 22:36:33 +00:00
eileencodes a8ea8bb4bd
Fix inferred query constraints
While working on implementing query constraints in our vitess gem, I
noticed that we were applying query constraints even when we didn't have
them on both sides of the association. We only want to infer query
constraints if both the `klass` and `active_record` have them set.

* If both the `klass` and `active_record` on the association don't have
query constraints set, use the `foreign_key` and don't infer query
constraints.
* Adds an error if the `active_record` and `klass` query constraints
don't match.
* Calling `klass` in `derive_fk_query_constraints` revealed that a bunch
of our tests are missing requires, without these we'll see test failures
that an association can't compute the klass name. Adding the requires
fixes those.
2023-09-12 11:57:51 -04:00
Gannon McGibbon e224768d14 Use id instead of number in composite key for Cpk::Book
Use [:author_id, :id] instead of [:author_id, :number] as
the primary key of Cpk::Book.
2023-06-26 15:33:28 -05:00
Joshua Young e856d7f1a2 [Fixes #48080] Fix broken distinct in `ActiveRecord::Calculations#ids` with `includes(...).order(...)` 2023-05-03 11:07:23 +10:00
Nikita Vasilevsky cce2a18eb9
[Tests only] Add coverage for `.count` calculations for a CPK model 2023-04-19 18:29:45 +00:00
Adrianna Chang 5ed3f60df6
Introduce adapter for Trilogy, a MySQL-compatible DB client
The [Trilogy database client][trilogy-client] and corresponding
[Active Record adapter][ar-adapter] were both open sourced by GitHub last year.

Shopify has recently taken the plunge and successfully adopted Trilogy in their Rails monolith.
With two major Rails applications running Trilogy successfully, we'd like to propose upstreaming the adapter
to Rails as a MySQL-compatible alternative to Mysql2Adapter.

[trilogy-client]: https://github.com/github/trilogy
[ar-adapter]: https://github.com/github/activerecord-trilogy-adapter

Co-authored-by: Aaron Patterson <tenderlove@github.com>
Co-authored-by: Adam Roben <adam@roben.org>
Co-authored-by: Ali Ibrahim <aibrahim2k2@gmail.com>
Co-authored-by: Aman Gupta <aman@tmm1.net>
Co-authored-by: Arthur Nogueira Neves <github@arthurnn.com>
Co-authored-by: Arthur Schreiber <arthurschreiber@github.com>
Co-authored-by: Ashe Connor <kivikakk@github.com>
Co-authored-by: Brandon Keepers <brandon@opensoul.org>
Co-authored-by: Brian Lopez <seniorlopez@gmail.com>
Co-authored-by: Brooke Kuhlmann <brooke@testdouble.com>
Co-authored-by: Bryana Knight <bryanaknight@github.com>
Co-authored-by: Carl Brasic <brasic@github.com>
Co-authored-by: Chris Bloom <chrisbloom7@github.com>
Co-authored-by: Cliff Pruitt <cliff.pruitt@cliffpruitt.com>
Co-authored-by: Daniel Colson <composerinteralia@github.com>
Co-authored-by: David Calavera <david.calavera@gmail.com>
Co-authored-by: David Celis <davidcelis@github.com>
Co-authored-by: David Ratajczak <david@mockra.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Eileen Uchitelle <eileencodes@gmail.com>
Co-authored-by: Enrique Gonzalez <enriikke@gmail.com>
Co-authored-by: Garrett Bjerkhoel <garrett@github.com>
Co-authored-by: Georgi Knox <georgicodes@github.com>
Co-authored-by: HParker <HParker@github.com>
Co-authored-by: Hailey Somerville <hailey@hailey.lol>
Co-authored-by: James Dennes <jdennes@gmail.com>
Co-authored-by: Jane Sternbach <janester@github.com>
Co-authored-by: Jess Bees <toomanybees@github.com>
Co-authored-by: Jesse Toth <jesse.toth@github.com>
Co-authored-by: Joel Hawksley <joelhawksley@github.com>
Co-authored-by: John Barnette <jbarnette@github.com>
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
Co-authored-by: John Hawthorn <john@hawthorn.email>
Co-authored-by: John Nunemaker <nunemaker@gmail.com>
Co-authored-by: Jonathan Hoyt <hoyt@github.com>
Co-authored-by: Katrina Owen <kytrinyx@github.com>
Co-authored-by: Keeran Raj Hawoldar <keeran@gmail.com>
Co-authored-by: Kevin Solorio <soloriok@gmail.com>
Co-authored-by: Leo Correa <lcorr005@gmail.com>
Co-authored-by: Lizz Hale <lizzhale@github.com>
Co-authored-by: Lorin Thwaits <lorint@gmail.com>
Co-authored-by: Matt Jones <al2o3cr@gmail.com>
Co-authored-by: Matthew Draper <matthewd@github.com>
Co-authored-by: Max Veytsman <mveytsman@github.com>
Co-authored-by: Nathan Witmer <nathan@zerowidth.com>
Co-authored-by: Nick Holden <nick.r.holden@gmail.com>
Co-authored-by: Paarth Madan <paarth.madan@shopify.com>
Co-authored-by: Patrick Reynolds <patrick.reynolds@github.com>
Co-authored-by: Rob Sanheim <rsanheim@gmail.com>
Co-authored-by: Rocio Delgado <rocio@github.com>
Co-authored-by: Sam Lambert <sam.lambert@github.com>
Co-authored-by: Shay Frendt <shay@github.com>
Co-authored-by: Shlomi Noach <shlomi-noach@github.com>
Co-authored-by: Sophie Haskins <sophaskins@github.com>
Co-authored-by: Thomas Maurer <tma@github.com>
Co-authored-by: Tim Pease <tim.pease@gmail.com>
Co-authored-by: Yossef Mendelssohn <ymendel@pobox.com>
Co-authored-by: Zack Koppert <zkoppert@github.com>
Co-authored-by: Zhongying Qiao <cryptoque@users.noreply.github.com>
2023-04-17 11:49:07 -04:00
fatkodima c8438fa998 Require missing CPK models in `base_test.rb` and `calculations_test.rb` 2023-04-13 19:49:16 +03:00
eileencodes 885bd850e3
Ensure that ids_writer and ids_reader is working for CPK
This fixes `ids_writer` so that it can handle a composite primary key.
Using a CPK model associated with a non-CPK model was working correctly
(which I added a test for). Using a CPK model associated with another
CPK model was not working correctly. It now takes it into account to
write the correct ids.

While working on `ids_reader` I found that `pluck` is not working for
CPK because it's passing an array of attributes and that's not supported
by `disallow_raw_sql!`. I chose to call `flatten` in `pluck` but not
conditionally because this seems like it could be a problem elsewhere as
well. This fixes pluck by CPK overall and fixes a test in the
calculations test file.
2023-04-12 15:02:58 -04:00
Nikita Vasilevsky 1217127703 Fix `ActiveRecord::Calculations#ids` for a composite primary key model
Given a model with a composite primary key like:
`TravelRoute.primary_key = [:from, :to]`

Calling `TravelRoute.ids` would return an array of identifiers which is
represented by an array of arrays:

```ruby
TravelRoute.all.ids # => [["Ottawa", "New York"], ["London", "Paris"]]
```
2023-03-24 12:45:54 +00:00
Rafael Mendonça França 3ec629784c
Remove deprecated override of `Enumerable#sum` 2023-03-03 00:38:26 +00:00
fatkodima cf940daa76 Fix ActiveRecord grouped calculations on joined tables on column present in both tables 2023-01-07 21:56:13 +02:00
Joshua Young 0c16feb4bc Update ActiveRecord::Calculations#ids and improve test coverage 2022-12-11 11:32:22 +05:30
Joshua Young 8fd336927a [Fix: #46455] ActiveRecord::Calculations#ids plucks included associations IDs 2022-12-09 08:23:48 +05:30
Jonathan Hefner 8a7aba0c76 Add missing assert_not_deprecated deprecator args
Follow-up to #46330 and #46337.

This adds a deprecator arg to some `assert_not_deprecated` calls that
were missed in #46330 and #46337.
2022-11-09 12:07:29 -06:00
Jonathan Hefner 682353e3de Add ActiveRecord.deprecator
This commit adds `ActiveRecord.deprecator` and replaces all usages of
`ActiveSupport::Deprecation.warn` in `activerecord/lib` with
`ActiveRecord.deprecator`.

Additionally, this commit adds `ActiveRecord.deprecator` to
`Rails.application.deprecators` so that it can be configured using e.g.
`config.active_support.report_deprecations`.
2022-10-25 14:23:08 -05:00
Dooor 31c15b4791 Fix a bug where using groups and counts with long table names would return incorrect results.
Fixed a bug that caused the alias name of "group by" to be too long and the first half of the name would be the same in both cases if it was cut by max identifier length.

Fix #46285

Co-authored-by: Yusaku ONO <yono@users.noreply.github.com>
2022-10-20 16:15:45 +09:00
Rafael Mendonça França 26ceb7fc33
Merge pull request #42933 from ghiculescu/postgres-column-types
Don't skip some columns in `column_types` on Postgres
2022-09-16 17:04:12 -04:00
Zack Deveau 611990f1a6
Change ActiveRecord::Coders::YAMLColumn default to safe_load
In Psych >= 4.0.0, load defaults to safe_load. This commit
makes the ActiveRecord::Coders::YAMLColum class use Psych safe_load
as the Rails default.

This default is configurable via ActiveRecord.use_yaml_unsafe_load

We conditionally fallback to the correct unsafe load if use_yaml_unsafe_load
is set to true. unsafe_load was introduced in Psych 4.0.0

The list of safe_load permitted classes is configurable via
ActiveRecord.yaml_column_permitted_classes

[CVE-2022-32224]
2022-07-12 09:26:46 -07:00
Luan Vieira a1f76dd712 Avoid query from calculations on contradictory relation
Previously calculations would make a query even when passed a
contradiction, like `User.where(id: []).count`. This commit optimzes
that to avoid an unnecessary query to the database.

Co-authored-by: John Hawthorn <jhawthorn@github.com>
Co-authored-by: Daniel Colson <composerinteralia@github.com>
2022-05-11 15:27:01 -07:00
Jean Boussier e5b08b9ebc Replace `async: true` parameter by `async_*` methods 2022-04-14 09:55:24 +02:00
Jean Boussier 03e435b943 PoC: Active Record API for general async queries
Followup: https://github.com/rails/rails/pull/41372

Something we knew we'd need when we implemented `Relation#load_async`
but that we chose to delay to have time to think about it.

Right now, Active Record async support is limited to "collection results",
but among the not so fast queries that would benefit from asynchronicity
you often find aggregates (e.g. `count`, `sum`, etc) as well as hand crafted
`find_by_sql` queries.

`load_async` was easy to add as an API, because `Relation` acts as a collection
so it was trivial to simply block whenever it was iterated while retaining total
API compatibility.

For aggregates and `find_by_sql`, we have no other choice but to return something
different in async mode, with its own API.

This proof of concept showcase what this API looks like for `Relation#count`:

```ruby
Post.where(published: true).count # => 2
promise = Post.where(published: true).count(async: true) # => #<ActiveRecord::Promise status=pending>
promise.value # => 2
```

This API should be applicable to all aggregate methods, as well as all methods
returning a single record, or anything other than a `Relation`.
2022-04-14 09:53:54 +02:00
Jean Boussier 7e6e9091e5 Properly quote autogenerated column aliases
Fix: https://github.com/rails/rails/issues/43878
2021-12-17 12:29:11 +01:00
Rafael Mendonça França 98f608bcf2
Remove deprecation for aggregations that are grouping in duplicated fields
Now we will apply the behavior that is intended to Rails 7, which ignores
the duplicated fields.
2021-11-19 21:43:15 +00:00
Ryuta Kamizono 46bb4dee09 Fix failing tests due to datetime with precision is not supported on MySQL 5.5 2021-10-14 18:18:54 +09:00
Alex Ghiculescu 587522e696 Don't skip some columns in `column_types` on Postgres
Fixes https://github.com/rails/rails/issues/41651, by partially reverting https://github.com/rails/rails/pull/39097

I ran the same benchmark as https://github.com/rails/rails/pull/39097 and it seems like this change does not cause a perf regression.

```ruby

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  gem "rails", path: "/Users/alex/code/rails" # github: "rails/rails", branch: "main"
  gem "sqlite3"
  gem "benchmark-ips"
end

require "active_record"

ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")

ActiveRecord::Schema.define do
  create_table :active_storage_blobs do |t|
    t.string   :key,          null: false
    t.string   :filename,     null: false
    t.string   :content_type
    t.text     :metadata
    t.string   :service_name, null: false
    t.bigint   :byte_size,    null: false
    t.string   :checksum,     null: false
    t.datetime :created_at,   null: false

    t.index [ :key ], unique: true
  end
end

class ActiveStorageBlob < ActiveRecord::Base
end

Benchmark.ips do |x|
  x.report("find_by") { ActiveStorageBlob.find_by(id: 1) }
end
```

This branch:

```
Warming up --------------------------------------
             find_by     1.940k i/100ms
Calculating -------------------------------------
             find_by     17.928k (± 4.8%) i/s -     91.180k in   5.098301s
```

Main:

```
Warming up --------------------------------------
             find_by     1.912k i/100ms
Calculating -------------------------------------
             find_by     17.961k (± 4.8%) i/s -     89.864k in   5.015252s
```
2021-09-15 10:51:54 -04:00
Jean Boussier e672575906 Allow Relation#sum to take an `init` parameters
This is needed to not prevent full use of `Enumerable#sum`
on `Relation` objects.
2021-05-10 13:37:25 +02:00
John Bampton 54e526e473 chore: fix grammar, spelling and minor whitespace fix 2021-04-13 21:35:50 +10:00
Ryuta Kamizono ea4a5a9406 Fix aggregate attribute on Enum types
To address the issues #39248 and #39271, #39255 and #39274 made
aggregated results type casted by the attribute types.

But I've realised that we could avoid enum mapping when implemented
#41431.

This change restores the expectation of #39039 especially in the Enum
case.

Fixes #41600.
2021-03-04 18:40:10 +09:00
Ryuta Kamizono e450a4a497 Fix `average` on decimal and enum attributes
Follow up to #40351 and #41418.

This fixes `average` on decimal and enum (on integer in general)
attributes to not do `type.deserialize`.

The precision and the scale on the column might be lower than the
calculated result.

And also, mapping the calculated result to enum label is quite
meaningless, the mapping result is almost nil.
2021-02-13 15:07:48 +09:00
Josua Schmid 1a9b84391f
Harmonize average aggregation with minimum and maximum
The real problem behind the previous implementation of average
aggregation was not that float columns returned `BigDecimal` but that
average skipped `ActiveModel` type casting.

This change introduces handling for the only needed special case
of average: integers. Now any fields based on
`ActiveRecord::Type::Integer` will be casted to `BigDecimal` when
aggregated with average.
2021-02-12 13:01:47 +01:00
Aaron Frase 91d583420b
Avoid stack level too deep in predicate builder
Check if `query` is different from `attributes` before recursively calling `expand_from_hash`.
Updated cases to account for new company and comment records.
2021-02-11 11:32:20 -05:00
Josua Schmid 78b43e6694
Return raw value for numeric AVG calculation results
After this change `float`-typed database columns will return `Float`
when aggregated with `AVG` (as expected and consistent with the
behaviour of `MIN` and `MAX`).

We keep returning `BigDecimal` for non-numeric column types which
respond to `to_d`.
2021-02-09 16:58:37 +01:00
Gannon McGibbon d4a9744860 Strip whitespace in disallow_raw_sql!
Removing trailing whitespace when matching columns in
`ActiveRecord::Sanitization.disallow_raw_sql!`.

Co-authored-by: Adrian Hirt <aedu.hirt@gmail.com>
2021-02-04 16:51:58 -05:00
Rafael Mendonça França 1b455e2e9d
Rails 6.2 is now Rails 7.0
We have big plans for the next version of Rails and that
require big versions.
2021-02-04 16:47:16 +00:00
Ryuta Kamizono 5259062868 Test calculation with `from` for all adapters 2020-09-09 19:37:58 +09:00
Eugene Kenny 5e2df1fc8e Add tests for loaded pluck and pick with alias 2020-06-30 10:50:43 +01:00
Ryuta Kamizono 9641d0b2ef Support type casting for grandchild's attributes
Related to #39292.

Fixes #39460.
2020-05-29 04:55:13 +09:00
Ryuta Kamizono 95e130524d Fix flaky assert queries tests
`assert_no_queries` sometimes fails due to counting implict schema info
load queries, most case that is enough to just ignore it.

https://buildkite.com/rails/rails/builds/69598#321bb1bc-ec67-40cd-813f-68dc7809ddde/1444-1452
2020-05-27 08:52:44 +09:00
Ryuta Kamizono 2580b83f42 Deprecate aggregations with group by duplicated fields
We've learned that `merge` causes duplicated multiple values easily, so
if we missed to deduplicate the values, it will cause weird behavior
like #38052, #39171.

I've investigated the deduplication for the values, at least that had
existed since Rails 3.0.

bed9179aa1

Aggregations with group by multiple fields was introduced at Rails 3.1,
but we missed the deduplication for the aggregation result, unlike the
generated SQL.

a5cdf0b9eb

While the investigation, I've found that `annotate` is also missed the
deduplication.

I don't suppose this weird behavior is intended for both.

So I'd like to deprecate the duplicated behavior in Rails 6.1, and will
be deduplicated all multiple values in Rails 6.2.

To migrate to Rails 6.2's behavior, use `uniq!(:group)` to deduplicate
group fields.

```ruby
accounts = Account.group(:firm_id)

# duplicated group fields, deprecated.
accounts.merge(accounts.where.not(credit_limit: nil)).sum(:credit_limit)
# => {
#   [1, 1] => 50,
#   [2, 2] => 60
# }

# use `uniq!(:group)` to deduplicate group fields.
accounts.merge(accounts.where.not(credit_limit: nil)).uniq!(:group).sum(:credit_limit)
# => {
#   1 => 50,
#   2 => 60
# }
```
2020-05-24 03:11:53 +09:00
Ryuta Kamizono af71a82265
Merge pull request #39297 from kamipo/fix_group_by_order_and_limit_offset
Fix group by count with eager loading + order + limit/offset
2020-05-16 18:42:26 +09:00
Ryuta Kamizono 5812feffe3 Fix group by count with eager loading + order + limit/offset
`count` is too complex feature in Active Record, it is heavily mangling
select values, so it easily hit the ORDER BY with SELECT DISTINCT
limitation.

35bf86fe83/activerecord/lib/active_record/relation/calculations.rb (L133-L140)
35bf86fe83/activerecord/lib/active_record/relation/calculations.rb (L253-L264)

But at least in the case of group by, select values are always to be
aggregated value and group values, so meaningful order values are
originally limited. So in that case, remaining order values should
be safe as long as meaningful order values are set by people.

Fixes #38936.
2020-05-15 17:16:59 +09:00
Ryuta Kamizono a89d20cfc7 Type cast aggregations values for table name unqalified column in joins tables
Follow up of #39292 and #39259.
2020-05-15 06:36:45 +09:00
Ryuta Kamizono 4b5d1e0670 Type cast `pluck` values for table name unqalified column in joins tables
Follow up to #39264, and fixes demonstrated case in #39290.

If the column has no type caster and the model don't know the attribute,
let's will attempt to lookup cast type from join dependency tree.
2020-05-15 05:16:23 +09:00
Ryuta Kamizono 89043b7f7f
Merge pull request #39264 from kamipo/fix_type_cast_pluck
Fix `pluck` to correctly type cast same column names and association columns
2020-05-14 06:22:19 +09:00