Commit Graph

661 Commits

Author SHA1 Message Date
Rafael França 9492339979
Merge pull request #38443 from jonathanhefner/plugin-fix-stack-traces
Fix stack traces when testing plugins
2020-10-08 19:06:40 -04:00
Eugene Kenny e44b3419d4 Prefix keys in Active Storage service test
Otherwise this test fails when run concurrently.

https://buildkite.com/rails/rails/builds/71999#77923632-f943-4fa0-b478-0af38001c22b/1032-1098
https://buildkite.com/rails/rails/builds/71999#01fdec79-dc1f-4a6a-b730-ac0df134a2e7/970-1062
2020-10-07 21:47:59 +01:00
Jonathan Hefner 33fdae0584 Fix backtraces for generated plugin tests
`Minitest.plugin_rails_init` sets `Minitest.backtrace_filter` to
`Rails.backtrace_cleaner` right before tests are run, overwriting the
value set in test_helper.rb.

`Rails.backtrace_cleaner` silences backtrace lines that do not start
with `Rails.root` followed by e.g. "lib/" or "test/".  Thus when
`Rails.root` is a subdirectory of the project directory -- for example,
when testing a plugin that has a dummy app -- all lines of the backtrace
are silenced.

This commit adds a fallback such that when all backtrace lines are
silenced, the original `Minitest.backtrace_filter` is used instead.

Additionally, this commit refactors and expands existing test coverage.
2020-10-07 15:40:56 -05:00
Akira Matsuda d71291cb3c Module#include is a public method since Ruby 2.1 2020-10-06 21:19:24 +09:00
Jonathan Hefner fd3fd618f4 Add gitattributes for vendored and generated files [ci-skip]
The `linguist-vendored` attribute excludes the specified file from the
project's language stats on GitHub.  The `linguist-generated` attribute
does the same, and also suppresses that file in diffs on GitHub.

See https://github.com/github/linguist for more information.
2020-10-04 16:44:37 -05:00
George Claghorn a50d2f1993 Extract ActiveStorage::Record
Permit applications to hack in custom DB config for ASt models until ASt has first-class multi-DB support:

    ActiveSupport.on_load(:active_storage_record) do
      connects_to reading: :active_storage_replica, writing: :active_storage_primary
    end

rebase
2020-09-29 11:59:14 -04:00
Pietro Moro 1e2cd7a75e
Support per-environment ASt service config 2020-09-29 09:59:52 -04:00
David Genord II f4aa54d487
Add requires for mimemagic
This dependency was recently added, but not the require. This looked like it worked because sometimes marcel does the require for us.
2020-09-25 15:00:04 -04:00
George Claghorn b6d86add73 Remove unused require
Follows up on a185ff8.
2020-09-23 14:39:52 -04:00
George Claghorn a185ff86c0
Set default variant format from blob content type 2020-09-22 13:03:53 -04:00
Vincent Robert 3803671a81 Generate a preview without print margins
When a PDF is used for both printing and displaying. It will most likely
contain a crop box in order to hide print margins when displaying the PDF.

Use Poppler's parameter to automatically use the crop box (visible box)
rather than the media box (printable box) in order to remove those margins
when drawing the PDF.

See https://manpages.debian.org/testing/poppler-utils/pdftoppm.1.en.html
2020-09-17 11:59:45 -04:00
Santiago Bartesaghi 4a78dcb326
Clarify ActiveStorage::Service#url docs [ci skip]
* Public services don’t respect `:disposition`, `:filename`, or `:content_type`.
* `:expires_in` is optional.
2020-09-12 12:47:33 -04:00
George Claghorn 9055156668 Override ActiveStorage.signed_id_verifier instead of assigning
Guard against the case where ActiveStorage.verifier isn't yet initialized at load time.

Yes, you're not supposed to load AR models in initializers, but it's easy to do accidentally as long as we don't prevent it. We should be resilient against it wherever practical.
2020-09-02 08:41:15 -04:00
George Claghorn b221a4dc43
Proxying: sanitize Content-Type and Content-Disposition
Prevent XSS where unsafe content is served inline on the application origin.

Follows up on #34477. References 06ab7b2 and d40284b.
2020-08-31 11:24:13 -04:00
Jonathan Hefner a197d39e13
Fall back to video container for duration
Some video formats, such as WebM, do not store duration information at
the level of the video stream.  Instead, the duration is stored as part
of the container format information.

This commit modifies `VideoAnalyzer` to use the duration from the video
container when the duration from the video stream is not available.

Fixes #40130.
2020-08-31 08:48:06 -04:00
Jonathan Hefner 601006c56d
Update service metadata for updated Blobs only
Follow-up to #40013.

Newly created `Blob`s may not be uploaded yet, so do not try to update
their service metadata.
2020-08-13 10:45:46 -04:00
Jonathan Hefner 6aa26c30e2
Identify directly-uploaded blobs before saving the associated record
An Active Storage `Blob` must be identified before it can be reliably
validated.  For direct uploads, a `Blob` is identified when it is
attached, rather than when it is created.

Before this commit, the sequence of events when attaching a `Blob` was:

1. Find the `Blob`.
2. Assign the `Blob` to an `Attachment`.
3. Save the owner record.
4. Save the `Attachment`.
5. Identify the `Blob`'s true `content_type` from its file.
6. Save the `Blob`.

This meant that the owner record's validations might not see the
`Blob`'s true `content_type`.

After this commit, the sequence of events will be:

1. Find the `Blob`.
2. Identify the `Blob`'s true `content_type` from its file.
3. Assign the `Blob` to an `Attachment`.
4. Save the owner record.
5. Save the `Attachment`.
6. Save the `Blob`.

Thus the `Blob`'s true `content_type` will be available when running the
owner record's validations.
2020-08-11 18:08:09 -04:00
Lawrence Chou eb5f5ed0be
Fix attaching blobs via nested attributes
Closes #37411.
2020-08-04 00:26:27 -04:00
Victor Perez Rodriguez 3eb48a2148 fix misleading variant test
This commit fixes the "resized variation of BMP blob" test.

By default `create_file_blob` use "image/jpeg" as content type,
since this test does not specify the correct `content_type` for
a `*.bmp` image ("image/bmp") the `ActiveStorage::Variant#specification`
consider the blob as a web image which causes the variant to return a
`*.bmp` URL and a "BMP" type, this is an incorrect behavior since if you
upload a `*.bmp` image the variant will have a PNG format with "image/png"
as content type.

After this change the test will cover the current activestorage behavior.

Changes:

* Specify correct `content_type` on "resized variation of BMP blob" test.
* Change asserts to cover the current activestorage behavior.
2020-07-29 16:32:14 -05:00
Victor Perez Rodriguez 396b43a99d touch parent model when an attachment is purged
Currently `delete` is used on the `purge` and `purge_later` methods,
that prevent any callbacks to be triggered which causes the parent
model to not be updated when an attachment is purged. This behaviour
cause issues on some caching strategies as reported here: https://github.com/rails/rails/issues/39858

Changes:

* Add `record&.touch` on `attachment#purge`
* Add `record&.touch` on `attachment#purge_later`
* Remove extra blank line on attachment.rb
* Add tests which are failing before this change and pass after the change
2020-07-26 12:22:37 -05:00
Evgeniy Rashchepkin 6af5662165
Disable template digesting for Active Storage controllers
Error entries appear in the log when we request ActiveStorage
controllers (`ActiveStorage::Representations::ProxyController#show`,
`ActiveStorage::Blobs::ProxyController#show`).

These entries look like: "Couldn't find template for digesting:
active_storage/representations/proxy/show".

These controllers use the method
`ActionController::ConditionalGet#http_cache_forever`,
and therefore `ActionController::ConditionalGet#combine_etags` method,
and therefore `ActionController::EtagWithTemplateDigest` module via
`etaggers` array.

`ActionController::EtagWithTemplateDigest` module requires a template
(view).

We have no views in ActiveStorage, so `EtagWithTemplateDigest` is
now turned off in ActiveStorage controllers by
`etag_with_template_digest` class attribute.
2020-07-05 10:51:08 -04:00
Santiago Bartesaghi 31148cd6be Use Active Record signed IDs in Active Storage 2020-07-04 21:09:00 -04:00
Ross Kaffenberger 04cbaa1475 Use ES module syntax for application.js.tt and docs
This change swaps the CommonJS require() syntax in the Webpacker
application.js pack template file and in documentation examples with ES
module import syntax.

Benefits of this change include:

Provides continuity with the larger frontend community: Arguably, one of
the main draws in adopting Webpacker is its integration with Babel to
support ES module syntax. For a fresh Rails install with Webpacker, the
application.js file will be the first impression most Rails developers
have with webpack and Webpacker.  Most of the recent documentation and
examples they will find online for using other libraries will be based
on ES module syntax.

Reduces confusion: Developers commonly add ES imports to their
application.js pack, typically by following online examples, which means
mixing require() and import statements in a single file. This leads to
confusion and unnecessary friction about differences between require()
and import.

Embraces browser-friendliness: The ES module syntax forward-looking and
is meant to be supported in browsers. On the other hand, require()
syntax is synchronous by design and not browser-supported as CommonJS
originally was adopted in Node.js for server-side JavaScript. That
webpack supports require() syntax is merely a convenience.

Encourages best practices regarding optimization: webpack can statically
analyze ES modules and "tree-shake", i.e., strip out unused exports from
the final build (given certain conditions are met, including
`sideEffects: false` designation in package.json).
2020-06-16 15:12:12 -04:00
George Claghorn fd61bae02c ActiveStorage::Attachment#signed_id must return a signed *blob* ID
Fixes that calling e.g. `url_for @user.avatar` would return an incorrect URL resulting in a 404. The Active Storage URL helper ends up calling ActiveStorage::Attachment#signed_id, which previously delegated to ActiveStorage::Blob#signed_id but changed to return a signed *attachment* ID in 1a3dc42.

The Active Storage controllers pass signed IDs to ActiveStorage::Blob.find_signed, so we need signed blob IDs.
2020-05-28 09:20:49 -04:00
Jonathan Hefner d02d259b61
Fix Active Storage behavior on record dup
Prior to this commit, a `dup`ed record and its originating record would
share the same `Attached` proxy objects.  Those proxy objects, in turn,
would point to the same `Attachment` associations, causing changes made
through the proxy interface to affect both records.
2020-05-25 22:03:32 -04:00
Jonathan Hefner 03f5f57af0 Test Attached::Many in Attached::Many test
This exact test exists in `one_test.rb`.  It was probably overlooked
after being copied and pasted.
2020-05-22 22:52:38 -05:00
George Claghorn 7e52d0ae58 Preserve old redirect URLs
They may be cached for existing apps upgrading to 6.1. Follows up on #34477.
2020-05-21 10:36:34 -04:00
fatkodima 6c4f3be929 Unify raise_on_missing_translations for views and controllers 2020-05-20 02:42:59 +03:00
Rafael França a80115b7c5
Merge pull request #39204 from prathamesh-sonpatki/template-annotation
Add the configuration option for annotating templates with file names to the generated app
2020-05-19 19:11:31 -04:00
Paul Blaze 84057dab16
Update aws-sdk-s3 dependency
whitelist_headers support added in 1.48.0:
https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-s3/CHANGELOG.md#1480-2019-08-30
2020-05-19 23:09:39 +08:00
Prathamesh Sonpatki a673ce69e7
Rename `annotate_template_file_names` to `annotate_rendered_view_with_filenames`
- Add the configuration option for annotating templates with file names to the generated app.
- Add `annotate_rendered_view_with_filenames` option to configuring guide.
2020-05-19 09:28:14 +05:30
Ryuta Kamizono 393df7425f Fix syntax error 2020-05-19 01:04:22 +09:00
Travis Pew c0ab9a7d29
Include Content-Length in signature for ActiveStorage direct upload
[CVE-2020-8162]
2020-05-15 14:32:19 -07:00
George Claghorn 57d926a78a Fix unintentional method redefinitions
References dfb5a82.
2020-05-14 09:46:16 -04:00
Jonathan Fleckenstein dfb5a82b25
Active Storage: allow serving files by proxying 2020-05-11 16:21:58 -04:00
Santiago Bartesaghi e61bdbf315
Make Active Storage update migrations reversible 2020-05-07 22:20:16 -04:00
Ryuta Kamizono 6a4395f466 Deprecate `starts_with?` and `ends_with?` for String core extensions
In the past, we sometimes hit missing `Symbol#start_with?` and
`Symbol#end_with?`.

63256bc5d7
a8e812964d

So I proposed `Symbol#start_with?` and `Symbol#end_with?` to allow duck
typing that methods for String and Symbol, then now it is available in
Ruby 2.7.

https://bugs.ruby-lang.org/issues/16348

Using `String#starts_with?` and `String#ends_with?` could not be gained
that conveniency, so it is preferable to not use these in the future.
2020-05-05 15:51:24 +09:00
fatkodima a1d960290d [ActiveStorage] Rewrite Blob.unattached scope code using where.missing 2020-05-04 01:01:30 +03:00
Xavier Noria c7c3abe03b removes require_dependency calls in ActiveStorage::Blob
The less we depend on require_dependency, the better.
2020-04-30 00:27:19 +02:00
George Claghorn 3a38c07211 Revert "Set a public ACL for files uploaded to a public GCS service"
This reverts commit 43503bdfec.
2020-04-28 17:16:47 -04:00
Brendan Abbott 43503bdfec
Set a public ACL for files uploaded to a public GCS service 2020-04-28 15:44:17 -04:00
Mikel Kew cf7c27f2ff
Ensure direct uploads to a public S3 service have a public ACL 2020-04-28 15:42:17 -04:00
Abhay Nikam 68846bdc1f
Verify FFmpeg presence before attempting to preview videos 2020-04-27 08:51:34 -04:00
William Carey 4c5e4ea476
Fix autosave association bug with ActiveStorage::Attachments
Closes #37701.
2020-04-24 13:16:16 -04:00
Godfrey Chan 1064c51609 Fix typos [ci skip]
I wrote this shell script to find words from the Rails repo,
so I can paste them into https://www.horsepaste.com/ for
the [codenames game](https://en.m.wikipedia.org/wiki/Codenames_(board_game)).

```bash
git grep -Il '' | \
  grep -v -E "CHANGELOG|Gemfile|gemspec|package\.json|yarn\.lock" | \
  xargs cat | \
  sed '/[^ ]\{10,\}/d' | \
  sed 's/\([A-Z]\)/ \1/g' | \
  tr 'A-Z' 'a-z' | \
  tr -c -s 'a-z' '\n' | \
  sed '/^.\{0,3\}$/d' | \
  sort | \
  uniq | \
  tr '\n' ',' | \
  pbcopy
```

You can see the result in https://www.horsepaste.com/rails-fixed.
Click "Next game" to cycle the words.

Found some typos in the codebase from this 😂

This is how I generated the list of possible typos:

```bash
git grep -Il '' | \
  grep -v -E "CHANGELOG|Gemfile|gemspec|package\.json|yarn\.lock" | \
  xargs cat | \
  sed '/[^ ]\{10,\}/d' | \
  sed 's/\([A-Z]\)/ \1/g' | \
  tr 'A-Z' 'a-z' | \
  tr -c -s 'a-z' '\n' | \
  sed '/^.\{0,3\}$/d' | \
  sort | \
  uniq | \
  aspell --ignore-case list
```

I manually reviewed the list and made the corrections
in this commit. The rest on the list are either:

* Bugs in my script: it split things like "doesn't" into
  "doesn" and "t", if find things like `#ffffff` and
  extracts "ffffff" as a word, etc
* British spelling: honour, optimised
* Foreign words: bonjour, espanol
* Names: nginx, hanekawa
* Technical words: mutex, xhtml
* Portmanteau words: autosave, nodelist
* Invented words: camelize, coachee
* Shortened words: attrs, repo
* Deliberate typos: hllo, hillo (used in code examples, etc)
* Lorem ipsum words: arcu, euismod

This is the [output](https://gist.github.com/chancancode/eb0b573d667dc31906f33f1fb0b22313)
of the script *after* fixing the typos included in this
commit. In theory, someone can run that command again in
the future and compare the output to catch new typos (i.e.
using my list to filter out known typos).

Limitations: the aspell dictionary could be wrong, I
could have miss things, and my script ignores words that
are less than 3 characters or longer than 10 characters.
2020-04-22 21:43:37 -07:00
JvH 2a12b723bb Add web_image_content_types config option for ActiveStorage
Add `config.active_storage.web_image_content_types` to allow applications
to add content types (like `image/webp`) in which variants can be processed,
instead of letting those images be converted to the fallback PNG format.
2020-04-21 14:00:41 -04:00
Ryuta Kamizono 6f2126c760 Fixup CHANGELOGs [ci skip] 2020-04-15 21:23:24 +09:00
Abhay Nikam 2b6754b9e4 Updated Active Storage configuration guide for previewers default values [ci skip] 2020-04-06 11:57:59 +05:30
Abhay Nikam bdfffd1355 Update the Rails mailing list URLs to new discuss discourse URL [ci skip] 2020-04-02 22:00:28 +05:30
Joel Hawksley a59e1de26a .annotate_template_file_names annotates HTML output with template file names
As a developer, when looking at a page in my web browser, it's sometimes
difficult to figure out which template(s) are being used to render the page.

config.action_view.annotate_template_file_names adds HTML comments to the
rendered output indicating where each template begins and ends.

Co-authored-by: Aaron Patterson <tenderlove@github.com>
2020-03-30 14:50:01 -06:00