License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 22:07:57 +08:00
|
|
|
// SPDX-License-Identifier: GPL-2.0
|
2006-10-11 16:20:50 +08:00
|
|
|
/*
|
2006-10-11 16:20:53 +08:00
|
|
|
* linux/fs/ext4/dir.c
|
2006-10-11 16:20:50 +08:00
|
|
|
*
|
|
|
|
* Copyright (C) 1992, 1993, 1994, 1995
|
|
|
|
* Remy Card (card@masi.ibp.fr)
|
|
|
|
* Laboratoire MASI - Institut Blaise Pascal
|
|
|
|
* Universite Pierre et Marie Curie (Paris VI)
|
|
|
|
*
|
|
|
|
* from
|
|
|
|
*
|
|
|
|
* linux/fs/minix/dir.c
|
|
|
|
*
|
|
|
|
* Copyright (C) 1991, 1992 Linus Torvalds
|
|
|
|
*
|
2006-10-11 16:20:53 +08:00
|
|
|
* ext4 directory handling functions
|
2006-10-11 16:20:50 +08:00
|
|
|
*
|
|
|
|
* Big-endian to little-endian byte-swapping/bitmaps by
|
|
|
|
* David S. Miller (davem@caip.rutgers.edu), 1995
|
|
|
|
*
|
|
|
|
* Hash Tree Directory indexing (c) 2001 Daniel Phillips
|
|
|
|
*
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include <linux/fs.h>
|
|
|
|
#include <linux/buffer_head.h>
|
|
|
|
#include <linux/slab.h>
|
2018-01-09 21:21:39 +08:00
|
|
|
#include <linux/iversion.h>
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-26 02:12:08 +08:00
|
|
|
#include <linux/unicode.h>
|
2008-04-30 06:13:32 +08:00
|
|
|
#include "ext4.h"
|
2012-12-11 03:05:59 +08:00
|
|
|
#include "xattr.h"
|
2006-10-11 16:20:50 +08:00
|
|
|
|
2013-05-18 04:08:53 +08:00
|
|
|
static int ext4_dx_readdir(struct file *, struct dir_context *);
|
2006-10-11 16:20:50 +08:00
|
|
|
|
2012-03-19 10:44:40 +08:00
|
|
|
/**
|
2019-06-20 04:30:03 +08:00
|
|
|
* is_dx_dir() - check if a directory is using htree indexing
|
|
|
|
* @inode: directory inode
|
|
|
|
*
|
2012-03-19 10:44:40 +08:00
|
|
|
* Check if the given dir-inode refers to an htree-indexed directory
|
2013-08-29 02:40:12 +08:00
|
|
|
* (or a directory which could potentially get converted to use htree
|
2012-03-19 10:44:40 +08:00
|
|
|
* indexing).
|
|
|
|
*
|
|
|
|
* Return 1 if it is a dx dir, 0 if not
|
|
|
|
*/
|
|
|
|
static int is_dx_dir(struct inode *inode)
|
|
|
|
{
|
|
|
|
struct super_block *sb = inode->i_sb;
|
|
|
|
|
2015-10-18 04:18:43 +08:00
|
|
|
if (ext4_has_feature_dir_index(inode->i_sb) &&
|
2012-03-19 10:44:40 +08:00
|
|
|
((ext4_test_inode_flag(inode, EXT4_INODE_INDEX)) ||
|
2013-04-20 05:53:09 +08:00
|
|
|
((inode->i_size >> sb->s_blocksize_bits) == 1) ||
|
|
|
|
ext4_has_inline_data(inode)))
|
2012-03-19 10:44:40 +08:00
|
|
|
return 1;
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2010-12-20 11:07:02 +08:00
|
|
|
/*
|
|
|
|
* Return 0 if the directory entry is OK, and 1 if there is a problem
|
|
|
|
*
|
|
|
|
* Note: this is the opposite of what ext2 and ext3 historically returned...
|
2012-12-11 03:05:59 +08:00
|
|
|
*
|
|
|
|
* bh passed here can be an inode block or a dir data block, depending
|
|
|
|
* on the inode inline data flag.
|
2010-12-20 11:07:02 +08:00
|
|
|
*/
|
2010-07-27 23:54:40 +08:00
|
|
|
int __ext4_check_dir_entry(const char *function, unsigned int line,
|
2011-01-11 01:10:55 +08:00
|
|
|
struct inode *dir, struct file *filp,
|
2010-07-27 23:54:40 +08:00
|
|
|
struct ext4_dir_entry_2 *de,
|
2012-12-11 03:05:58 +08:00
|
|
|
struct buffer_head *bh, char *buf, int size,
|
2010-07-27 23:54:40 +08:00
|
|
|
unsigned int offset)
|
2006-10-11 16:20:50 +08:00
|
|
|
{
|
2008-09-09 10:25:24 +08:00
|
|
|
const char *error_msg = NULL;
|
2009-02-15 12:01:36 +08:00
|
|
|
const int rlen = ext4_rec_len_from_disk(de->rec_len,
|
|
|
|
dir->i_sb->s_blocksize);
|
2019-12-09 08:43:46 +08:00
|
|
|
const int next_offset = ((char *) de - buf) + rlen;
|
2006-10-11 16:20:50 +08:00
|
|
|
|
2010-12-20 11:07:02 +08:00
|
|
|
if (unlikely(rlen < EXT4_DIR_REC_LEN(1)))
|
2006-10-11 16:20:50 +08:00
|
|
|
error_msg = "rec_len is smaller than minimal";
|
2010-12-20 11:07:02 +08:00
|
|
|
else if (unlikely(rlen % 4 != 0))
|
2006-10-11 16:20:50 +08:00
|
|
|
error_msg = "rec_len % 4 != 0";
|
2010-12-20 11:07:02 +08:00
|
|
|
else if (unlikely(rlen < EXT4_DIR_REC_LEN(de->name_len)))
|
2006-10-11 16:20:50 +08:00
|
|
|
error_msg = "rec_len is too small for name_len";
|
2019-12-09 08:43:46 +08:00
|
|
|
else if (unlikely(next_offset > size))
|
2018-08-27 21:22:45 +08:00
|
|
|
error_msg = "directory entry overrun";
|
2019-12-09 08:43:46 +08:00
|
|
|
else if (unlikely(next_offset > size - EXT4_DIR_REC_LEN(1) &&
|
|
|
|
next_offset != size))
|
2019-12-03 01:02:13 +08:00
|
|
|
error_msg = "directory entry too close to block end";
|
2010-12-20 11:07:02 +08:00
|
|
|
else if (unlikely(le32_to_cpu(de->inode) >
|
|
|
|
le32_to_cpu(EXT4_SB(dir->i_sb)->s_es->s_inodes_count)))
|
2006-10-11 16:20:50 +08:00
|
|
|
error_msg = "inode out of bounds";
|
2010-12-20 11:07:02 +08:00
|
|
|
else
|
|
|
|
return 0;
|
|
|
|
|
2011-01-11 01:10:55 +08:00
|
|
|
if (filp)
|
2012-02-21 06:53:05 +08:00
|
|
|
ext4_error_file(filp, function, line, bh->b_blocknr,
|
2018-08-27 21:22:45 +08:00
|
|
|
"bad entry in directory: %s - offset=%u, "
|
|
|
|
"inode=%u, rec_len=%d, name_len=%d, size=%d",
|
|
|
|
error_msg, offset, le32_to_cpu(de->inode),
|
|
|
|
rlen, de->name_len, size);
|
2011-01-11 01:10:55 +08:00
|
|
|
else
|
2012-02-21 06:53:05 +08:00
|
|
|
ext4_error_inode(dir, function, line, bh->b_blocknr,
|
2018-08-27 21:22:45 +08:00
|
|
|
"bad entry in directory: %s - offset=%u, "
|
|
|
|
"inode=%u, rec_len=%d, name_len=%d, size=%d",
|
|
|
|
error_msg, offset, le32_to_cpu(de->inode),
|
|
|
|
rlen, de->name_len, size);
|
2011-01-11 01:10:55 +08:00
|
|
|
|
2010-12-20 11:07:02 +08:00
|
|
|
return 1;
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
|
|
|
|
2013-05-18 04:08:53 +08:00
|
|
|
static int ext4_readdir(struct file *file, struct dir_context *ctx)
|
2006-10-11 16:20:50 +08:00
|
|
|
{
|
2008-11-05 13:14:04 +08:00
|
|
|
unsigned int offset;
|
2014-05-28 00:48:55 +08:00
|
|
|
int i;
|
2006-10-11 16:20:53 +08:00
|
|
|
struct ext4_dir_entry_2 *de;
|
2006-10-11 16:20:50 +08:00
|
|
|
int err;
|
2013-05-18 04:08:53 +08:00
|
|
|
struct inode *inode = file_inode(file);
|
2012-03-19 10:44:40 +08:00
|
|
|
struct super_block *sb = inode->i_sb;
|
2015-04-12 13:09:05 +08:00
|
|
|
struct buffer_head *bh = NULL;
|
2016-07-11 02:01:03 +08:00
|
|
|
struct fscrypt_str fstr = FSTR_INIT(NULL, 0);
|
2006-10-11 16:20:50 +08:00
|
|
|
|
2018-12-12 17:50:10 +08:00
|
|
|
if (IS_ENCRYPTED(inode)) {
|
2016-07-11 02:01:03 +08:00
|
|
|
err = fscrypt_get_encryption_info(inode);
|
2019-12-10 05:23:48 +08:00
|
|
|
if (err)
|
2016-02-08 08:35:05 +08:00
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2012-03-19 10:44:40 +08:00
|
|
|
if (is_dx_dir(inode)) {
|
2013-05-18 04:08:53 +08:00
|
|
|
err = ext4_dx_readdir(file, ctx);
|
2006-10-11 16:20:50 +08:00
|
|
|
if (err != ERR_BAD_DX_DIR) {
|
2013-05-18 04:08:53 +08:00
|
|
|
return err;
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
2020-02-10 22:43:16 +08:00
|
|
|
/* Can we just clear INDEX flag to ignore htree information? */
|
|
|
|
if (!ext4_has_metadata_csum(sb)) {
|
|
|
|
/*
|
|
|
|
* We don't set the inode dirty flag since it's not
|
|
|
|
* critical that it gets flushed back to the disk.
|
|
|
|
*/
|
|
|
|
ext4_clear_inode_flag(inode, EXT4_INODE_INDEX);
|
|
|
|
}
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
2013-04-20 05:53:09 +08:00
|
|
|
|
|
|
|
if (ext4_has_inline_data(inode)) {
|
|
|
|
int has_inline_data = 1;
|
2015-04-12 13:09:05 +08:00
|
|
|
err = ext4_read_inline_dir(file, ctx,
|
2013-04-20 05:53:09 +08:00
|
|
|
&has_inline_data);
|
|
|
|
if (has_inline_data)
|
2015-04-12 13:09:05 +08:00
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2018-12-12 17:50:10 +08:00
|
|
|
if (IS_ENCRYPTED(inode)) {
|
2020-08-10 22:21:39 +08:00
|
|
|
err = fscrypt_fname_alloc_buffer(EXT4_NAME_LEN, &fstr);
|
ext4 crypto: reorganize how we store keys in the inode
This is a pretty massive patch which does a number of different things:
1) The per-inode encryption information is now stored in an allocated
data structure, ext4_crypt_info, instead of directly in the node.
This reduces the size usage of an in-memory inode when it is not
using encryption.
2) We drop the ext4_fname_crypto_ctx entirely, and use the per-inode
encryption structure instead. This remove an unnecessary memory
allocation and free for the fname_crypto_ctx as well as allowing us
to reuse the ctfm in a directory for multiple lookups and file
creations.
3) We also cache the inode's policy information in the ext4_crypt_info
structure so we don't have to continually read it out of the
extended attributes.
4) We now keep the keyring key in the inode's encryption structure
instead of releasing it after we are done using it to derive the
per-inode key. This allows us to test to see if the key has been
revoked; if it has, we prevent the use of the derived key and free
it.
5) When an inode is released (or when the derived key is freed), we
will use memset_explicit() to zero out the derived key, so it's not
left hanging around in memory. This implies that when a user logs
out, it is important to first revoke the key, and then unlink it,
and then finally, to use "echo 3 > /proc/sys/vm/drop_caches" to
release any decrypted pages and dcache entries from the system
caches.
6) All this, and we also shrink the number of lines of code by around
100. :-)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-05-19 01:17:47 +08:00
|
|
|
if (err < 0)
|
2015-04-12 13:09:05 +08:00
|
|
|
return err;
|
2013-04-20 05:53:09 +08:00
|
|
|
}
|
|
|
|
|
2013-05-18 04:08:53 +08:00
|
|
|
while (ctx->pos < inode->i_size) {
|
2010-05-17 08:00:00 +08:00
|
|
|
struct ext4_map_blocks map;
|
2006-10-11 16:20:50 +08:00
|
|
|
|
2016-04-24 10:50:07 +08:00
|
|
|
if (fatal_signal_pending(current)) {
|
|
|
|
err = -ERESTARTSYS;
|
|
|
|
goto errout;
|
|
|
|
}
|
|
|
|
cond_resched();
|
2019-06-21 09:19:02 +08:00
|
|
|
offset = ctx->pos & (sb->s_blocksize - 1);
|
2013-05-18 04:08:53 +08:00
|
|
|
map.m_lblk = ctx->pos >> EXT4_BLOCK_SIZE_BITS(sb);
|
2010-05-17 08:00:00 +08:00
|
|
|
map.m_len = 1;
|
|
|
|
err = ext4_map_blocks(NULL, inode, &map, 0);
|
2019-06-21 09:19:02 +08:00
|
|
|
if (err == 0) {
|
|
|
|
/* m_len should never be zero but let's avoid
|
|
|
|
* an infinite loop if it somehow is */
|
|
|
|
if (map.m_len == 0)
|
|
|
|
map.m_len = 1;
|
|
|
|
ctx->pos += map.m_len * sb->s_blocksize;
|
|
|
|
continue;
|
|
|
|
}
|
2006-10-11 16:20:50 +08:00
|
|
|
if (err > 0) {
|
2010-05-17 08:00:00 +08:00
|
|
|
pgoff_t index = map.m_pblk >>
|
mm, fs: get rid of PAGE_CACHE_* and page_cache_{get,release} macros
PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} macros were introduced *long* time
ago with promise that one day it will be possible to implement page
cache with bigger chunks than PAGE_SIZE.
This promise never materialized. And unlikely will.
We have many places where PAGE_CACHE_SIZE assumed to be equal to
PAGE_SIZE. And it's constant source of confusion on whether
PAGE_CACHE_* or PAGE_* constant should be used in a particular case,
especially on the border between fs and mm.
Global switching to PAGE_CACHE_SIZE != PAGE_SIZE would cause to much
breakage to be doable.
Let's stop pretending that pages in page cache are special. They are
not.
The changes are pretty straight-forward:
- <foo> << (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- <foo> >> (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} -> PAGE_{SIZE,SHIFT,MASK,ALIGN};
- page_cache_get() -> get_page();
- page_cache_release() -> put_page();
This patch contains automated changes generated with coccinelle using
script below. For some reason, coccinelle doesn't patch header files.
I've called spatch for them manually.
The only adjustment after coccinelle is revert of changes to
PAGE_CAHCE_ALIGN definition: we are going to drop it later.
There are few places in the code where coccinelle didn't reach. I'll
fix them manually in a separate patch. Comments and documentation also
will be addressed with the separate patch.
virtual patch
@@
expression E;
@@
- E << (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
expression E;
@@
- E >> (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
@@
- PAGE_CACHE_SHIFT
+ PAGE_SHIFT
@@
@@
- PAGE_CACHE_SIZE
+ PAGE_SIZE
@@
@@
- PAGE_CACHE_MASK
+ PAGE_MASK
@@
expression E;
@@
- PAGE_CACHE_ALIGN(E)
+ PAGE_ALIGN(E)
@@
expression E;
@@
- page_cache_get(E)
+ get_page(E)
@@
expression E;
@@
- page_cache_release(E)
+ put_page(E)
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-04-01 20:29:47 +08:00
|
|
|
(PAGE_SHIFT - inode->i_blkbits);
|
2013-05-18 04:08:53 +08:00
|
|
|
if (!ra_has_index(&file->f_ra, index))
|
2007-07-19 16:48:08 +08:00
|
|
|
page_cache_sync_readahead(
|
2007-07-19 16:48:04 +08:00
|
|
|
sb->s_bdev->bd_inode->i_mapping,
|
2013-05-18 04:08:53 +08:00
|
|
|
&file->f_ra, file,
|
2007-07-19 16:48:08 +08:00
|
|
|
index, 1);
|
mm, fs: get rid of PAGE_CACHE_* and page_cache_{get,release} macros
PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} macros were introduced *long* time
ago with promise that one day it will be possible to implement page
cache with bigger chunks than PAGE_SIZE.
This promise never materialized. And unlikely will.
We have many places where PAGE_CACHE_SIZE assumed to be equal to
PAGE_SIZE. And it's constant source of confusion on whether
PAGE_CACHE_* or PAGE_* constant should be used in a particular case,
especially on the border between fs and mm.
Global switching to PAGE_CACHE_SIZE != PAGE_SIZE would cause to much
breakage to be doable.
Let's stop pretending that pages in page cache are special. They are
not.
The changes are pretty straight-forward:
- <foo> << (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- <foo> >> (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} -> PAGE_{SIZE,SHIFT,MASK,ALIGN};
- page_cache_get() -> get_page();
- page_cache_release() -> put_page();
This patch contains automated changes generated with coccinelle using
script below. For some reason, coccinelle doesn't patch header files.
I've called spatch for them manually.
The only adjustment after coccinelle is revert of changes to
PAGE_CAHCE_ALIGN definition: we are going to drop it later.
There are few places in the code where coccinelle didn't reach. I'll
fix them manually in a separate patch. Comments and documentation also
will be addressed with the separate patch.
virtual patch
@@
expression E;
@@
- E << (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
expression E;
@@
- E >> (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
@@
- PAGE_CACHE_SHIFT
+ PAGE_SHIFT
@@
@@
- PAGE_CACHE_SIZE
+ PAGE_SIZE
@@
@@
- PAGE_CACHE_MASK
+ PAGE_MASK
@@
expression E;
@@
- PAGE_CACHE_ALIGN(E)
+ PAGE_ALIGN(E)
@@
expression E;
@@
- page_cache_get(E)
+ get_page(E)
@@
expression E;
@@
- page_cache_release(E)
+ put_page(E)
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-04-01 20:29:47 +08:00
|
|
|
file->f_ra.prev_pos = (loff_t)index << PAGE_SHIFT;
|
2014-08-30 08:52:15 +08:00
|
|
|
bh = ext4_bread(NULL, inode, map.m_lblk, 0);
|
2016-02-16 13:20:19 +08:00
|
|
|
if (IS_ERR(bh)) {
|
|
|
|
err = PTR_ERR(bh);
|
|
|
|
bh = NULL;
|
|
|
|
goto errout;
|
|
|
|
}
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
if (!bh) {
|
[PATCH] handle ext4 directory corruption better
I've been using Steve Grubb's purely evil "fsfuzzer" tool, at
http://people.redhat.com/sgrubb/files/fsfuzzer-0.4.tar.gz
Basically it makes a filesystem, splats some random bits over it, then
tries to mount it and do some simple filesystem actions.
At best, the filesystem catches the corruption gracefully. At worst,
things spin out of control.
As you might guess, we found a couple places in ext4 where things spin out
of control :)
First, we had a corrupted directory that was never checked for
consistency... it was corrupt, and pointed to another bad "entry" of
length 0. The for() loop looped forever, since the length of
ext4_next_entry(de) was 0, and we kept looking at the same pointer over and
over and over and over... I modeled this check and subsequent action on
what is done for other directory types in ext4_readdir...
(adding this check adds some computational expense; I am testing a followup
patch to reduce the number of times we check and re-check these directory
entries, in all cases. Thanks for the idea, Andreas).
Next we had a root directory inode which had a corrupted size, claimed to
be > 200M on a 4M filesystem. There was only really 1 block in the
directory, but because the size was so large, readdir kept coming back for
more, spewing thousands of printk's along the way.
Per Andreas' suggestion, if we're in this read error condition and we're
trying to read an offset which is greater than i_blocks worth of bytes,
stop trying, and break out of the loop.
With these two changes fsfuzz test survives quite well on ext4.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Cc: <linux-ext4@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-07 12:36:28 +08:00
|
|
|
/* corrupt size? Maybe no more blocks to read */
|
2013-05-18 04:08:53 +08:00
|
|
|
if (ctx->pos > inode->i_blocks << 9)
|
[PATCH] handle ext4 directory corruption better
I've been using Steve Grubb's purely evil "fsfuzzer" tool, at
http://people.redhat.com/sgrubb/files/fsfuzzer-0.4.tar.gz
Basically it makes a filesystem, splats some random bits over it, then
tries to mount it and do some simple filesystem actions.
At best, the filesystem catches the corruption gracefully. At worst,
things spin out of control.
As you might guess, we found a couple places in ext4 where things spin out
of control :)
First, we had a corrupted directory that was never checked for
consistency... it was corrupt, and pointed to another bad "entry" of
length 0. The for() loop looped forever, since the length of
ext4_next_entry(de) was 0, and we kept looking at the same pointer over and
over and over and over... I modeled this check and subsequent action on
what is done for other directory types in ext4_readdir...
(adding this check adds some computational expense; I am testing a followup
patch to reduce the number of times we check and re-check these directory
entries, in all cases. Thanks for the idea, Andreas).
Next we had a root directory inode which had a corrupted size, claimed to
be > 200M on a 4M filesystem. There was only really 1 block in the
directory, but because the size was so large, readdir kept coming back for
more, spewing thousands of printk's along the way.
Per Andreas' suggestion, if we're in this read error condition and we're
trying to read an offset which is greater than i_blocks worth of bytes,
stop trying, and break out of the loop.
With these two changes fsfuzz test survives quite well on ext4.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Cc: <linux-ext4@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-07 12:36:28 +08:00
|
|
|
break;
|
2013-05-18 04:08:53 +08:00
|
|
|
ctx->pos += sb->s_blocksize - offset;
|
2006-10-11 16:20:50 +08:00
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
2012-04-30 06:41:10 +08:00
|
|
|
/* Check the checksum */
|
|
|
|
if (!buffer_verified(bh) &&
|
2019-06-22 03:49:26 +08:00
|
|
|
!ext4_dirblock_csum_verify(inode, bh)) {
|
2013-05-18 04:08:53 +08:00
|
|
|
EXT4_ERROR_FILE(file, 0, "directory fails checksum "
|
2012-04-30 06:41:10 +08:00
|
|
|
"at offset %llu",
|
2013-05-18 04:08:53 +08:00
|
|
|
(unsigned long long)ctx->pos);
|
|
|
|
ctx->pos += sb->s_blocksize - offset;
|
2013-01-29 10:23:24 +08:00
|
|
|
brelse(bh);
|
2015-04-12 13:09:05 +08:00
|
|
|
bh = NULL;
|
2012-04-30 06:41:10 +08:00
|
|
|
continue;
|
|
|
|
}
|
|
|
|
set_buffer_verified(bh);
|
|
|
|
|
2006-10-11 16:20:50 +08:00
|
|
|
/* If the dir block has changed since the last call to
|
|
|
|
* readdir(2), then we might be pointing to an invalid
|
|
|
|
* dirent right now. Scan from the start of the block
|
|
|
|
* to make sure. */
|
2018-02-01 21:15:25 +08:00
|
|
|
if (!inode_eq_iversion(inode, file->f_version)) {
|
2006-10-11 16:20:50 +08:00
|
|
|
for (i = 0; i < sb->s_blocksize && i < offset; ) {
|
2006-10-11 16:20:53 +08:00
|
|
|
de = (struct ext4_dir_entry_2 *)
|
2006-10-11 16:20:50 +08:00
|
|
|
(bh->b_data + i);
|
|
|
|
/* It's too expensive to do a full
|
|
|
|
* dirent test each time round this
|
|
|
|
* loop, but we do have to test at
|
|
|
|
* least that it is non-zero. A
|
|
|
|
* failure will be detected in the
|
|
|
|
* dirent test below. */
|
2009-02-15 12:01:36 +08:00
|
|
|
if (ext4_rec_len_from_disk(de->rec_len,
|
|
|
|
sb->s_blocksize) < EXT4_DIR_REC_LEN(1))
|
2006-10-11 16:20:50 +08:00
|
|
|
break;
|
2009-02-15 12:01:36 +08:00
|
|
|
i += ext4_rec_len_from_disk(de->rec_len,
|
|
|
|
sb->s_blocksize);
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
|
|
|
offset = i;
|
2013-05-18 04:08:53 +08:00
|
|
|
ctx->pos = (ctx->pos & ~(sb->s_blocksize - 1))
|
2006-10-11 16:20:50 +08:00
|
|
|
| offset;
|
2018-01-09 21:21:39 +08:00
|
|
|
file->f_version = inode_query_iversion(inode);
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
|
|
|
|
2013-05-18 04:08:53 +08:00
|
|
|
while (ctx->pos < inode->i_size
|
2006-10-11 16:20:50 +08:00
|
|
|
&& offset < sb->s_blocksize) {
|
2006-10-11 16:20:53 +08:00
|
|
|
de = (struct ext4_dir_entry_2 *) (bh->b_data + offset);
|
2013-05-18 04:08:53 +08:00
|
|
|
if (ext4_check_dir_entry(inode, file, de, bh,
|
2012-12-11 03:05:58 +08:00
|
|
|
bh->b_data, bh->b_size,
|
|
|
|
offset)) {
|
2006-10-11 16:21:24 +08:00
|
|
|
/*
|
2013-05-18 04:08:53 +08:00
|
|
|
* On error, skip to the next block
|
2006-10-11 16:21:24 +08:00
|
|
|
*/
|
2013-05-18 04:08:53 +08:00
|
|
|
ctx->pos = (ctx->pos |
|
2006-10-11 16:20:50 +08:00
|
|
|
(sb->s_blocksize - 1)) + 1;
|
2013-05-18 04:08:53 +08:00
|
|
|
break;
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
2009-02-15 12:01:36 +08:00
|
|
|
offset += ext4_rec_len_from_disk(de->rec_len,
|
|
|
|
sb->s_blocksize);
|
2006-10-11 16:20:50 +08:00
|
|
|
if (le32_to_cpu(de->inode)) {
|
2018-12-12 17:50:10 +08:00
|
|
|
if (!IS_ENCRYPTED(inode)) {
|
2015-04-12 13:09:05 +08:00
|
|
|
if (!dir_emit(ctx, de->name,
|
|
|
|
de->name_len,
|
|
|
|
le32_to_cpu(de->inode),
|
|
|
|
get_dtype(sb, de->file_type)))
|
|
|
|
goto done;
|
|
|
|
} else {
|
2016-07-11 02:01:03 +08:00
|
|
|
int save_len = fstr.len;
|
|
|
|
struct fscrypt_str de_name =
|
|
|
|
FSTR_INIT(de->name,
|
|
|
|
de->name_len);
|
2015-05-19 01:15:47 +08:00
|
|
|
|
2015-04-12 13:09:05 +08:00
|
|
|
/* Directory is encrypted */
|
2016-07-11 02:01:03 +08:00
|
|
|
err = fscrypt_fname_disk_to_usr(inode,
|
|
|
|
0, 0, &de_name, &fstr);
|
2016-09-16 05:25:55 +08:00
|
|
|
de_name = fstr;
|
2016-07-11 02:01:03 +08:00
|
|
|
fstr.len = save_len;
|
2016-09-16 05:25:55 +08:00
|
|
|
if (err)
|
2015-04-12 13:09:05 +08:00
|
|
|
goto errout;
|
|
|
|
if (!dir_emit(ctx,
|
2016-09-16 05:25:55 +08:00
|
|
|
de_name.name, de_name.len,
|
2015-04-12 13:09:05 +08:00
|
|
|
le32_to_cpu(de->inode),
|
|
|
|
get_dtype(sb, de->file_type)))
|
|
|
|
goto done;
|
2013-05-18 04:08:53 +08:00
|
|
|
}
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
2013-05-18 04:08:53 +08:00
|
|
|
ctx->pos += ext4_rec_len_from_disk(de->rec_len,
|
2009-02-15 12:01:36 +08:00
|
|
|
sb->s_blocksize);
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
2016-05-13 08:36:01 +08:00
|
|
|
if ((ctx->pos < inode->i_size) && !dir_relax_shared(inode))
|
2015-04-12 13:09:05 +08:00
|
|
|
goto done;
|
2008-09-09 10:25:24 +08:00
|
|
|
brelse(bh);
|
2015-04-12 13:09:05 +08:00
|
|
|
bh = NULL;
|
|
|
|
offset = 0;
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
2015-04-12 13:09:05 +08:00
|
|
|
done:
|
|
|
|
err = 0;
|
|
|
|
errout:
|
2016-07-11 02:01:03 +08:00
|
|
|
fscrypt_fname_free_buffer(&fstr);
|
2015-04-12 13:09:05 +08:00
|
|
|
brelse(bh);
|
|
|
|
return err;
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
|
|
|
|
2012-03-19 10:44:40 +08:00
|
|
|
static inline int is_32bit_api(void)
|
|
|
|
{
|
|
|
|
#ifdef CONFIG_COMPAT
|
2016-03-23 05:25:04 +08:00
|
|
|
return in_compat_syscall();
|
2012-03-19 10:44:40 +08:00
|
|
|
#else
|
|
|
|
return (BITS_PER_LONG == 32);
|
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
2006-10-11 16:20:50 +08:00
|
|
|
/*
|
|
|
|
* These functions convert from the major/minor hash to an f_pos
|
2012-03-19 10:44:40 +08:00
|
|
|
* value for dx directories
|
|
|
|
*
|
|
|
|
* Upper layer (for example NFS) should specify FMODE_32BITHASH or
|
|
|
|
* FMODE_64BITHASH explicitly. On the other hand, we allow ext4 to be mounted
|
|
|
|
* directly on both 32-bit and 64-bit nodes, under such case, neither
|
|
|
|
* FMODE_32BITHASH nor FMODE_64BITHASH is specified.
|
|
|
|
*/
|
|
|
|
static inline loff_t hash2pos(struct file *filp, __u32 major, __u32 minor)
|
|
|
|
{
|
|
|
|
if ((filp->f_mode & FMODE_32BITHASH) ||
|
|
|
|
(!(filp->f_mode & FMODE_64BITHASH) && is_32bit_api()))
|
|
|
|
return major >> 1;
|
|
|
|
else
|
|
|
|
return ((__u64)(major >> 1) << 32) | (__u64)minor;
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline __u32 pos2maj_hash(struct file *filp, loff_t pos)
|
|
|
|
{
|
|
|
|
if ((filp->f_mode & FMODE_32BITHASH) ||
|
|
|
|
(!(filp->f_mode & FMODE_64BITHASH) && is_32bit_api()))
|
|
|
|
return (pos << 1) & 0xffffffff;
|
|
|
|
else
|
|
|
|
return ((pos >> 32) << 1) & 0xffffffff;
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline __u32 pos2min_hash(struct file *filp, loff_t pos)
|
|
|
|
{
|
|
|
|
if ((filp->f_mode & FMODE_32BITHASH) ||
|
|
|
|
(!(filp->f_mode & FMODE_64BITHASH) && is_32bit_api()))
|
|
|
|
return 0;
|
|
|
|
else
|
|
|
|
return pos & 0xffffffff;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Return 32- or 64-bit end-of-file for dx directories
|
|
|
|
*/
|
|
|
|
static inline loff_t ext4_get_htree_eof(struct file *filp)
|
|
|
|
{
|
|
|
|
if ((filp->f_mode & FMODE_32BITHASH) ||
|
|
|
|
(!(filp->f_mode & FMODE_64BITHASH) && is_32bit_api()))
|
|
|
|
return EXT4_HTREE_EOF_32BIT;
|
|
|
|
else
|
|
|
|
return EXT4_HTREE_EOF_64BIT;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
2012-05-01 02:14:03 +08:00
|
|
|
* ext4_dir_llseek() calls generic_file_llseek_size to handle htree
|
|
|
|
* directories, where the "offset" is in terms of the filename hash
|
|
|
|
* value instead of the byte offset.
|
2006-10-11 16:20:50 +08:00
|
|
|
*
|
2012-05-01 02:14:03 +08:00
|
|
|
* Because we may return a 64-bit hash that is well beyond offset limits,
|
|
|
|
* we need to pass the max hash as the maximum allowable offset in
|
|
|
|
* the htree directory case.
|
|
|
|
*
|
|
|
|
* For non-htree, ext4_llseek already chooses the proper max offset.
|
2006-10-11 16:20:50 +08:00
|
|
|
*/
|
2013-03-03 06:24:05 +08:00
|
|
|
static loff_t ext4_dir_llseek(struct file *file, loff_t offset, int whence)
|
2012-03-19 10:44:40 +08:00
|
|
|
{
|
|
|
|
struct inode *inode = file->f_mapping->host;
|
|
|
|
int dx_dir = is_dx_dir(inode);
|
2018-04-02 11:21:03 +08:00
|
|
|
loff_t ret, htree_max = ext4_get_htree_eof(file);
|
2012-03-19 10:44:40 +08:00
|
|
|
|
2012-05-01 02:14:03 +08:00
|
|
|
if (likely(dx_dir))
|
2018-04-02 11:21:03 +08:00
|
|
|
ret = generic_file_llseek_size(file, offset, whence,
|
2012-05-01 02:14:03 +08:00
|
|
|
htree_max, htree_max);
|
|
|
|
else
|
2018-04-02 11:21:03 +08:00
|
|
|
ret = ext4_llseek(file, offset, whence);
|
|
|
|
file->f_version = inode_peek_iversion(inode) - 1;
|
|
|
|
return ret;
|
2012-03-19 10:44:40 +08:00
|
|
|
}
|
2006-10-11 16:20:50 +08:00
|
|
|
|
|
|
|
/*
|
|
|
|
* This structure holds the nodes of the red-black tree used to store
|
|
|
|
* the directory entry in hash order.
|
|
|
|
*/
|
|
|
|
struct fname {
|
|
|
|
__u32 hash;
|
|
|
|
__u32 minor_hash;
|
|
|
|
struct rb_node rb_hash;
|
|
|
|
struct fname *next;
|
|
|
|
__u32 inode;
|
|
|
|
__u8 name_len;
|
|
|
|
__u8 file_type;
|
2020-03-09 23:48:38 +08:00
|
|
|
char name[];
|
2006-10-11 16:20:50 +08:00
|
|
|
};
|
|
|
|
|
|
|
|
/*
|
|
|
|
* This functoin implements a non-recursive way of freeing all of the
|
|
|
|
* nodes in the red-black tree.
|
|
|
|
*/
|
|
|
|
static void free_rb_tree_fname(struct rb_root *root)
|
|
|
|
{
|
2014-01-24 07:56:10 +08:00
|
|
|
struct fname *fname, *next;
|
|
|
|
|
|
|
|
rbtree_postorder_for_each_entry_safe(fname, next, root, rb_hash)
|
2006-10-11 16:20:50 +08:00
|
|
|
while (fname) {
|
2008-09-09 10:25:24 +08:00
|
|
|
struct fname *old = fname;
|
2006-10-11 16:20:50 +08:00
|
|
|
fname = fname->next;
|
2008-09-09 10:25:24 +08:00
|
|
|
kfree(old);
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
2014-01-24 07:56:10 +08:00
|
|
|
|
|
|
|
*root = RB_ROOT;
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
|
2012-03-19 10:44:40 +08:00
|
|
|
static struct dir_private_info *ext4_htree_create_dir_info(struct file *filp,
|
|
|
|
loff_t pos)
|
2006-10-11 16:20:50 +08:00
|
|
|
{
|
|
|
|
struct dir_private_info *p;
|
|
|
|
|
2017-08-25 01:50:24 +08:00
|
|
|
p = kzalloc(sizeof(*p), GFP_KERNEL);
|
2006-10-11 16:20:50 +08:00
|
|
|
if (!p)
|
|
|
|
return NULL;
|
2012-03-19 10:44:40 +08:00
|
|
|
p->curr_hash = pos2maj_hash(filp, pos);
|
|
|
|
p->curr_minor_hash = pos2min_hash(filp, pos);
|
2006-10-11 16:20:50 +08:00
|
|
|
return p;
|
|
|
|
}
|
|
|
|
|
2006-10-11 16:20:53 +08:00
|
|
|
void ext4_htree_free_dir_info(struct dir_private_info *p)
|
2006-10-11 16:20:50 +08:00
|
|
|
{
|
|
|
|
free_rb_tree_fname(&p->root);
|
|
|
|
kfree(p);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Given a directory entry, enter it into the fname rb tree.
|
2015-04-12 12:56:26 +08:00
|
|
|
*
|
|
|
|
* When filename encryption is enabled, the dirent will hold the
|
|
|
|
* encrypted filename, while the htree will hold decrypted filename.
|
|
|
|
* The decrypted filename is passed in via ent_name. parameter.
|
2006-10-11 16:20:50 +08:00
|
|
|
*/
|
2006-10-11 16:20:53 +08:00
|
|
|
int ext4_htree_store_dirent(struct file *dir_file, __u32 hash,
|
2006-10-11 16:20:50 +08:00
|
|
|
__u32 minor_hash,
|
2015-04-12 12:56:26 +08:00
|
|
|
struct ext4_dir_entry_2 *dirent,
|
2016-07-11 02:01:03 +08:00
|
|
|
struct fscrypt_str *ent_name)
|
2006-10-11 16:20:50 +08:00
|
|
|
{
|
|
|
|
struct rb_node **p, *parent = NULL;
|
2008-09-09 10:25:24 +08:00
|
|
|
struct fname *fname, *new_fn;
|
2006-10-11 16:20:50 +08:00
|
|
|
struct dir_private_info *info;
|
|
|
|
int len;
|
|
|
|
|
2010-07-27 23:56:04 +08:00
|
|
|
info = dir_file->private_data;
|
2006-10-11 16:20:50 +08:00
|
|
|
p = &info->root.rb_node;
|
|
|
|
|
|
|
|
/* Create and allocate the fname structure */
|
2015-04-12 12:56:26 +08:00
|
|
|
len = sizeof(struct fname) + ent_name->len + 1;
|
2006-10-11 16:20:50 +08:00
|
|
|
new_fn = kzalloc(len, GFP_KERNEL);
|
|
|
|
if (!new_fn)
|
|
|
|
return -ENOMEM;
|
|
|
|
new_fn->hash = hash;
|
|
|
|
new_fn->minor_hash = minor_hash;
|
|
|
|
new_fn->inode = le32_to_cpu(dirent->inode);
|
2015-04-12 12:56:26 +08:00
|
|
|
new_fn->name_len = ent_name->len;
|
2006-10-11 16:20:50 +08:00
|
|
|
new_fn->file_type = dirent->file_type;
|
2015-04-12 12:56:26 +08:00
|
|
|
memcpy(new_fn->name, ent_name->name, ent_name->len);
|
2006-10-11 16:20:50 +08:00
|
|
|
|
|
|
|
while (*p) {
|
|
|
|
parent = *p;
|
|
|
|
fname = rb_entry(parent, struct fname, rb_hash);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If the hash and minor hash match up, then we put
|
|
|
|
* them on a linked list. This rarely happens...
|
|
|
|
*/
|
|
|
|
if ((new_fn->hash == fname->hash) &&
|
|
|
|
(new_fn->minor_hash == fname->minor_hash)) {
|
|
|
|
new_fn->next = fname->next;
|
|
|
|
fname->next = new_fn;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (new_fn->hash < fname->hash)
|
|
|
|
p = &(*p)->rb_left;
|
|
|
|
else if (new_fn->hash > fname->hash)
|
|
|
|
p = &(*p)->rb_right;
|
|
|
|
else if (new_fn->minor_hash < fname->minor_hash)
|
|
|
|
p = &(*p)->rb_left;
|
|
|
|
else /* if (new_fn->minor_hash > fname->minor_hash) */
|
|
|
|
p = &(*p)->rb_right;
|
|
|
|
}
|
|
|
|
|
|
|
|
rb_link_node(&new_fn->rb_hash, parent, p);
|
|
|
|
rb_insert_color(&new_fn->rb_hash, &info->root);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
2006-10-11 16:20:53 +08:00
|
|
|
* This is a helper function for ext4_dx_readdir. It calls filldir
|
2006-10-11 16:20:50 +08:00
|
|
|
* for all entres on the fname linked list. (Normally there is only
|
|
|
|
* one entry on the linked list, unless there are 62 bit hash collisions.)
|
|
|
|
*/
|
2013-05-18 04:08:53 +08:00
|
|
|
static int call_filldir(struct file *file, struct dir_context *ctx,
|
|
|
|
struct fname *fname)
|
2006-10-11 16:20:50 +08:00
|
|
|
{
|
2013-05-18 04:08:53 +08:00
|
|
|
struct dir_private_info *info = file->private_data;
|
|
|
|
struct inode *inode = file_inode(file);
|
|
|
|
struct super_block *sb = inode->i_sb;
|
2006-10-11 16:20:50 +08:00
|
|
|
|
|
|
|
if (!fname) {
|
2012-03-20 11:41:49 +08:00
|
|
|
ext4_msg(sb, KERN_ERR, "%s:%d: inode #%lu: comm %s: "
|
|
|
|
"called with null fname?!?", __func__, __LINE__,
|
|
|
|
inode->i_ino, current->comm);
|
2006-10-11 16:20:50 +08:00
|
|
|
return 0;
|
|
|
|
}
|
2013-05-18 04:08:53 +08:00
|
|
|
ctx->pos = hash2pos(file, fname->hash, fname->minor_hash);
|
2006-10-11 16:20:50 +08:00
|
|
|
while (fname) {
|
2013-05-18 04:08:53 +08:00
|
|
|
if (!dir_emit(ctx, fname->name,
|
|
|
|
fname->name_len,
|
2006-10-11 16:20:50 +08:00
|
|
|
fname->inode,
|
2013-05-18 04:08:53 +08:00
|
|
|
get_dtype(sb, fname->file_type))) {
|
2008-08-20 09:57:43 +08:00
|
|
|
info->extra_fname = fname;
|
2013-05-18 04:08:53 +08:00
|
|
|
return 1;
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
|
|
|
fname = fname->next;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2013-05-18 04:08:53 +08:00
|
|
|
static int ext4_dx_readdir(struct file *file, struct dir_context *ctx)
|
2006-10-11 16:20:50 +08:00
|
|
|
{
|
2013-05-18 04:08:53 +08:00
|
|
|
struct dir_private_info *info = file->private_data;
|
|
|
|
struct inode *inode = file_inode(file);
|
2006-10-11 16:20:50 +08:00
|
|
|
struct fname *fname;
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
if (!info) {
|
2013-05-18 04:08:53 +08:00
|
|
|
info = ext4_htree_create_dir_info(file, ctx->pos);
|
2006-10-11 16:20:50 +08:00
|
|
|
if (!info)
|
|
|
|
return -ENOMEM;
|
2013-05-18 04:08:53 +08:00
|
|
|
file->private_data = info;
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
|
|
|
|
2013-05-18 04:08:53 +08:00
|
|
|
if (ctx->pos == ext4_get_htree_eof(file))
|
2006-10-11 16:20:50 +08:00
|
|
|
return 0; /* EOF */
|
|
|
|
|
|
|
|
/* Some one has messed with f_pos; reset the world */
|
2013-05-18 04:08:53 +08:00
|
|
|
if (info->last_pos != ctx->pos) {
|
2006-10-11 16:20:50 +08:00
|
|
|
free_rb_tree_fname(&info->root);
|
|
|
|
info->curr_node = NULL;
|
|
|
|
info->extra_fname = NULL;
|
2013-05-18 04:08:53 +08:00
|
|
|
info->curr_hash = pos2maj_hash(file, ctx->pos);
|
|
|
|
info->curr_minor_hash = pos2min_hash(file, ctx->pos);
|
2006-10-11 16:20:50 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If there are any leftover names on the hash collision
|
|
|
|
* chain, return them first.
|
|
|
|
*/
|
2008-08-20 09:57:43 +08:00
|
|
|
if (info->extra_fname) {
|
2013-05-18 04:08:53 +08:00
|
|
|
if (call_filldir(file, ctx, info->extra_fname))
|
2008-08-20 09:57:43 +08:00
|
|
|
goto finished;
|
|
|
|
info->extra_fname = NULL;
|
2008-10-25 23:39:08 +08:00
|
|
|
goto next_node;
|
2008-08-20 09:57:43 +08:00
|
|
|
} else if (!info->curr_node)
|
2006-10-11 16:20:50 +08:00
|
|
|
info->curr_node = rb_first(&info->root);
|
|
|
|
|
|
|
|
while (1) {
|
|
|
|
/*
|
|
|
|
* Fill the rbtree if we have no more entries,
|
|
|
|
* or the inode has changed since we last read in the
|
|
|
|
* cached entries.
|
|
|
|
*/
|
|
|
|
if ((!info->curr_node) ||
|
2018-02-01 21:15:25 +08:00
|
|
|
!inode_eq_iversion(inode, file->f_version)) {
|
2006-10-11 16:20:50 +08:00
|
|
|
info->curr_node = NULL;
|
|
|
|
free_rb_tree_fname(&info->root);
|
2018-01-09 21:21:39 +08:00
|
|
|
file->f_version = inode_query_iversion(inode);
|
2013-05-18 04:08:53 +08:00
|
|
|
ret = ext4_htree_fill_tree(file, info->curr_hash,
|
2006-10-11 16:20:50 +08:00
|
|
|
info->curr_minor_hash,
|
|
|
|
&info->next_hash);
|
|
|
|
if (ret < 0)
|
|
|
|
return ret;
|
|
|
|
if (ret == 0) {
|
2013-05-18 04:08:53 +08:00
|
|
|
ctx->pos = ext4_get_htree_eof(file);
|
2006-10-11 16:20:50 +08:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
info->curr_node = rb_first(&info->root);
|
|
|
|
}
|
|
|
|
|
|
|
|
fname = rb_entry(info->curr_node, struct fname, rb_hash);
|
|
|
|
info->curr_hash = fname->hash;
|
|
|
|
info->curr_minor_hash = fname->minor_hash;
|
2013-05-18 04:08:53 +08:00
|
|
|
if (call_filldir(file, ctx, fname))
|
2006-10-11 16:20:50 +08:00
|
|
|
break;
|
2008-10-25 23:39:08 +08:00
|
|
|
next_node:
|
2006-10-11 16:20:50 +08:00
|
|
|
info->curr_node = rb_next(info->curr_node);
|
2008-10-25 23:39:08 +08:00
|
|
|
if (info->curr_node) {
|
|
|
|
fname = rb_entry(info->curr_node, struct fname,
|
|
|
|
rb_hash);
|
|
|
|
info->curr_hash = fname->hash;
|
|
|
|
info->curr_minor_hash = fname->minor_hash;
|
|
|
|
} else {
|
2006-10-11 16:20:50 +08:00
|
|
|
if (info->next_hash == ~0) {
|
2013-05-18 04:08:53 +08:00
|
|
|
ctx->pos = ext4_get_htree_eof(file);
|
2006-10-11 16:20:50 +08:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
info->curr_hash = info->next_hash;
|
|
|
|
info->curr_minor_hash = 0;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
finished:
|
2013-05-18 04:08:53 +08:00
|
|
|
info->last_pos = ctx->pos;
|
2006-10-11 16:20:50 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2015-06-01 01:34:57 +08:00
|
|
|
static int ext4_dir_open(struct inode * inode, struct file * filp)
|
|
|
|
{
|
2018-12-12 17:50:10 +08:00
|
|
|
if (IS_ENCRYPTED(inode))
|
2016-07-11 02:01:03 +08:00
|
|
|
return fscrypt_get_encryption_info(inode) ? -EACCES : 0;
|
2015-06-01 01:34:57 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-09-09 10:25:24 +08:00
|
|
|
static int ext4_release_dir(struct inode *inode, struct file *filp)
|
2006-10-11 16:20:50 +08:00
|
|
|
{
|
2006-10-11 16:21:24 +08:00
|
|
|
if (filp->private_data)
|
2006-10-11 16:20:53 +08:00
|
|
|
ext4_htree_free_dir_info(filp->private_data);
|
2006-10-11 16:20:50 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
2012-03-19 10:44:40 +08:00
|
|
|
|
2014-07-29 01:06:26 +08:00
|
|
|
int ext4_check_all_de(struct inode *dir, struct buffer_head *bh, void *buf,
|
|
|
|
int buf_size)
|
|
|
|
{
|
|
|
|
struct ext4_dir_entry_2 *de;
|
2016-09-30 14:14:56 +08:00
|
|
|
int rlen;
|
2014-07-29 01:06:26 +08:00
|
|
|
unsigned int offset = 0;
|
|
|
|
char *top;
|
|
|
|
|
|
|
|
de = (struct ext4_dir_entry_2 *)buf;
|
|
|
|
top = buf + buf_size;
|
|
|
|
while ((char *) de < top) {
|
|
|
|
if (ext4_check_dir_entry(dir, NULL, de, bh,
|
|
|
|
buf, buf_size, offset))
|
2015-10-18 04:16:04 +08:00
|
|
|
return -EFSCORRUPTED;
|
2014-07-29 01:06:26 +08:00
|
|
|
rlen = ext4_rec_len_from_disk(de->rec_len, buf_size);
|
|
|
|
de = (struct ext4_dir_entry_2 *)((char *)de + rlen);
|
|
|
|
offset += rlen;
|
|
|
|
}
|
|
|
|
if ((char *) de > top)
|
2015-10-18 04:16:04 +08:00
|
|
|
return -EFSCORRUPTED;
|
2014-07-29 01:06:26 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-03-19 10:44:40 +08:00
|
|
|
const struct file_operations ext4_dir_operations = {
|
|
|
|
.llseek = ext4_dir_llseek,
|
|
|
|
.read = generic_read_dir,
|
2016-05-13 08:36:01 +08:00
|
|
|
.iterate_shared = ext4_readdir,
|
2012-03-19 10:44:40 +08:00
|
|
|
.unlocked_ioctl = ext4_ioctl,
|
|
|
|
#ifdef CONFIG_COMPAT
|
|
|
|
.compat_ioctl = ext4_compat_ioctl,
|
|
|
|
#endif
|
|
|
|
.fsync = ext4_sync_file,
|
2015-06-01 01:34:57 +08:00
|
|
|
.open = ext4_dir_open,
|
2012-03-19 10:44:40 +08:00
|
|
|
.release = ext4_release_dir,
|
|
|
|
};
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-26 02:12:08 +08:00
|
|
|
|
|
|
|
#ifdef CONFIG_UNICODE
|
|
|
|
static int ext4_d_compare(const struct dentry *dentry, unsigned int len,
|
|
|
|
const char *str, const struct qstr *name)
|
|
|
|
{
|
|
|
|
struct qstr qstr = {.name = str, .len = len };
|
2020-01-24 12:12:34 +08:00
|
|
|
const struct dentry *parent = READ_ONCE(dentry->d_parent);
|
2020-10-10 16:10:16 +08:00
|
|
|
const struct inode *inode = d_inode_rcu(parent);
|
2020-06-02 04:05:43 +08:00
|
|
|
char strbuf[DNAME_INLINE_LEN];
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-26 02:12:08 +08:00
|
|
|
|
2020-01-24 12:12:34 +08:00
|
|
|
if (!inode || !IS_CASEFOLDED(inode) ||
|
|
|
|
!EXT4_SB(inode->i_sb)->s_encoding) {
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-26 02:12:08 +08:00
|
|
|
if (len != name->len)
|
|
|
|
return -1;
|
2019-05-25 11:48:23 +08:00
|
|
|
return memcmp(str, name->name, len);
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-26 02:12:08 +08:00
|
|
|
}
|
|
|
|
|
2020-06-02 04:05:43 +08:00
|
|
|
/*
|
|
|
|
* If the dentry name is stored in-line, then it may be concurrently
|
|
|
|
* modified by a rename. If this happens, the VFS will eventually retry
|
|
|
|
* the lookup, so it doesn't matter what ->d_compare() returns.
|
|
|
|
* However, it's unsafe to call utf8_strncasecmp() with an unstable
|
|
|
|
* string. Therefore, we have to copy the name into a temporary buffer.
|
|
|
|
*/
|
|
|
|
if (len <= DNAME_INLINE_LEN - 1) {
|
|
|
|
memcpy(strbuf, str, len);
|
|
|
|
strbuf[len] = 0;
|
|
|
|
qstr.name = strbuf;
|
|
|
|
/* prevent compiler from optimizing out the temporary buffer */
|
|
|
|
barrier();
|
|
|
|
}
|
|
|
|
|
2019-09-03 13:43:17 +08:00
|
|
|
return ext4_ci_compare(inode, name, &qstr, false);
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-26 02:12:08 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
static int ext4_d_hash(const struct dentry *dentry, struct qstr *str)
|
|
|
|
{
|
|
|
|
const struct ext4_sb_info *sbi = EXT4_SB(dentry->d_sb);
|
|
|
|
const struct unicode_map *um = sbi->s_encoding;
|
2020-10-10 16:10:16 +08:00
|
|
|
const struct inode *inode = d_inode_rcu(dentry);
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-26 02:12:08 +08:00
|
|
|
unsigned char *norm;
|
|
|
|
int len, ret = 0;
|
|
|
|
|
2020-01-24 12:12:34 +08:00
|
|
|
if (!inode || !IS_CASEFOLDED(inode) || !um)
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-26 02:12:08 +08:00
|
|
|
return 0;
|
|
|
|
|
|
|
|
norm = kmalloc(PATH_MAX, GFP_ATOMIC);
|
|
|
|
if (!norm)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
len = utf8_casefold(um, str, norm, PATH_MAX);
|
|
|
|
if (len < 0) {
|
|
|
|
if (ext4_has_strict_mode(sbi))
|
|
|
|
ret = -EINVAL;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
str->hash = full_name_hash(dentry, norm, len);
|
|
|
|
out:
|
|
|
|
kfree(norm);
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
const struct dentry_operations ext4_dentry_ops = {
|
|
|
|
.d_hash = ext4_d_hash,
|
|
|
|
.d_compare = ext4_d_compare,
|
|
|
|
};
|
|
|
|
#endif
|