Previously, mdBook used JavaScript to add header links, so we
skipped checking the book. As of
https://github.com/rust-lang/rust/pull/39966, it no longer does,
so we can start checking again.
There is a twist, though: it uses name instead of id, so let's test
for both. They're both valid links anyway, so it's good to have the
checker check anyway.
This reverts commit 7f1d1c6d9a.
The original commit was created because mdBook and rustdoc had
different generation algorithms for header links; now with
https://github.com/rust-lang/rust/pull/39966 , the algorithms
are the same. So let's undo this change.
... when I came across this problem, I said "eh, this isn't fun,
but it doesn't take that long." I probably should have just actually
taken the time to fix upstream, given that they were amenable. Oh
well!
Update mdbook version
This version of mdbook includes
https://github.com/azerupi/mdBook/pull/207 , which is needed so that
we can start doing linkchecker on the various books.
Docs: Better explanation of return values for min, max functions for the Iterator trait
Added an explanation that `None` is returned if an iterator is empty.
Also added examples for `max` and `min`. I chose not to add examples for other functions like `max_by_key` etc. so that the examples stay concised and focused on the main functionality.
Fixup String::split_off documentation
1. Clarify that `String::split_off` returns one string and modifies self in-place. The documentation implied that it returns two new strings.
2. Make the documentation mirror `Vec::split_off`.
Correct a typo in procedural macros chapter of the Book.
A simple and short correction for procedural macros chapter of the Rust Programming Language Book.
erase late bound regions in `get_vtable_methods()`
Higher-ranked object types can otherwise cause late-bound regions to
sneak into the substs, leading to the false conclusion that some method
is unreachable.
r? @arielb1, who wrote the heart of this patch anyhow
Fixes#39292
[MIR] Make InlineAsm a Statement
Previously InlineAsm was an Rvalue, but its semantics doesn't really match the semantics of an
Rvalue - rather it behaves more like a Statement.
r? @nikomatsakis you wanted this to happen
rustc: Link statically to the MSVCRT
This commit changes all MSVC rustc binaries to be compiled with
`-C target-feature=+crt-static` to link statically against the MSVCRT instead of
dynamically (as it does today). This also necessitates compiling LLVM in a
different fashion, ensuring it's compiled with `/MT` instead of `/MD`.
cc #37406
Fix for bootstrapping on NixOS
NixOS puts Linux's dynamic loader in wierd place. Detect when we're on NixOS and patch the downloaded bootstrap executables appropriately.
appveyor: Move MSVC dist builds to their own builder
In the long run we want to separate out the dist builders from the test
builders. This provides us leeway to expand the dist builders with more tools
(e.g. Cargo and the RLS) without impacting cycle times.
Currently the Travis dist builders double-up the platforms they provide builds
for, so I figured we could try that out for MSVC as well. This commit adds a new
AppVeyor builder which runs a dist for all the MSVC targets:
* x86_64-pc-windows-msvc
* i686-pc-windows-msvc
* i586-pc-windows-msvc
If this takes too long and/or times out we'll need to split this up. In any case
we're going to need more capacity from AppVeyor no matter what becaue the two
pc-windows-gnu targets can't cross compile so we need at least 2 more builders
no matter what.
1. Clarify that `String::split_off` returns one string and modifies self
in-place. The documentation implied that it returns two new strings.
2. Make the documentation mirror `Vec::split_off`.
Adaptive hashmap implementation
All credits to @pczarn who wrote https://github.com/rust-lang/rfcs/pull/1796 and https://github.com/contain-rs/hashmap2/pull/5
**Background**
Rust std lib hashmap puts a strong emphasis on security, we did some improvements in https://github.com/rust-lang/rust/pull/37470 but in some very specific cases and for non-default hashers it's still vulnerable (see #36481).
This is a simplified version of https://github.com/rust-lang/rfcs/pull/1796 proposal sans switching hashers on the fly and other things that require an RFC process and further decisions. I think this part has great potential by itself.
**Proposal**
This PR adds code checking for extra long probe and shifts lengths (see code comments and https://github.com/rust-lang/rfcs/pull/1796 for details), when those are encountered the hashmap will grow (even if the capacity limit is not reached yet) _greatly_ attenuating the degenerate performance case.
We need a lower bound on the minimum occupancy that may trigger the early resize, otherwise in extreme cases it's possible to turn the CPU attack into a memory attack. The PR code puts that lower bound at half of the max occupancy (defined by ResizePolicy). This reduces the protection (it could potentially be exploited between 0-50% occupancy) but makes it completely safe.
**Drawbacks**
* May interact badly with poor hashers. Maps using those may not use the desired capacity.
* It adds 2-3 branches to the common insert path, luckily those are highly predictable and there's room to shave some in future patches.
* May complicate exposure of ResizePolicy in the future as the constants are a function of the fill factor.
**Example**
Example code that exploit the exposure of iteration order and weak hasher.
```
const MERGE: usize = 10_000usize;
#[bench]
fn merge_dos(b: &mut Bencher) {
let first_map: $hashmap<usize, usize, FnvBuilder> = (0..MERGE).map(|i| (i, i)).collect();
let second_map: $hashmap<usize, usize, FnvBuilder> = (MERGE..MERGE * 2).map(|i| (i, i)).collect();
b.iter(|| {
let mut merged = first_map.clone();
for (&k, &v) in &second_map {
merged.insert(k, v);
}
::test::black_box(merged);
});
}
```
_91 is stdlib and _ad is patched (the end capacity in both cases is the same)
```
running 2 tests
test _91::merge_dos ... bench: 47,311,843 ns/iter (+/- 2,040,302)
test _ad::merge_dos ... bench: 599,099 ns/iter (+/- 83,270)
```
In the long run we want to separate out the dist builders from the test
builders. This provides us leeway to expand the dist builders with more tools
(e.g. Cargo and the RLS) without impacting cycle times.
Currently the Travis dist builders double-up the platforms they provide builds
for, so I figured we could try that out for MSVC as well. This commit adds a new
AppVeyor builder which runs a dist for all the MSVC targets:
* x86_64-pc-windows-msvc
* i686-pc-windows-msvc
* i586-pc-windows-msvc
If this takes too long and/or times out we'll need to split this up. In any case
we're going to need more capacity from AppVeyor no matter what becaue the two
pc-windows-gnu targets can't cross compile so we need at least 2 more builders
no matter what.
Higher-ranked object types can otherwise cause late-bound regions to
sneak into the substs, leading to the false conclusion that some method
is unreachable. The heart of this patch is from @arielb1.
travis: Disable source tarballs on most builders
Currently we create a source tarball on almost all of the `DEPLOY=1` builders
but this has the adverse side effect of all source tarballs overriding
themselves in the S3 bucket. Normally this is ok but unfortunately a source
tarball created on Windows is not buildable on Unix.
On Windows the vendored sources contain paths with `\` characters in them which
when interpreted on Unix end up in "file not found" errors.
Instead of this overwriting behavior, whitelist just one linux builder for
producing tarballs and avoid producing tarballs on all other hosts.
use bash when invoking dist shell scripts on solaris
Partially fixes#25845
A separate, trivial fix is needed to the rust-installer scripts to completely resolve this issue.
sys/mod doc update and mod import order adjust
* Some doc updates.
* Racer currently use the first mod it finds regardless of cfg attrs. Moving #[cfg(unix)] up should be a temporary tweak that works as expected for more people.
Vec, LinkedList, VecDeque, String, and Option NatVis visualizations
I've added some basic [NatVis](https://msdn.microsoft.com/en-us/library/jj620914.aspx) visualizations for core Rust collections and types. This helps address a need filed in issue #36503. NatVis visualizations are similar to gdb/lldb pretty printers, but for windbg and the Visual Studio debugger on Windows.
For example, Vec without the supplied NatVis looks like this in windbg using the "dx" command:
```
0:000> dx some_64_bit_vec
some_64_bit_vec [Type: collections::vec::Vec<u64>]
[+0x000] buf [Type: alloc::raw_vec::RawVec<u64>]
[+0x010] len : 0x4 [Type: unsigned __int64]
```
With the NatVis, the elements of the Vec are displayed:
```
0:000> dx some_64_bit_vec
some_64_bit_vec : { size=0x4 } [Type: collections::vec::Vec<u64>]
[<Raw View>] [Type: collections::vec::Vec<u64>]
[size] : 0x4 [Type: unsigned __int64]
[capacity] : 0x4 [Type: unsigned __int64]
[0] : 0x4 [Type: unsigned __int64]
[1] : 0x4f [Type: unsigned __int64]
[2] : 0x1a [Type: unsigned __int64]
[3] : 0x184 [Type: unsigned __int64]
```
In fact, the vector can be treated as an array by the NatVis expression evaluator:
```
0:000> dx some_64_bit_vec[2]
some_64_bit_vec[2] : 0x1a [Type: unsigned __int64]
```
In general, it works with any NatVis command that understands collections, such as NatVis LINQ expressions:
```
0:000> dx some_64_bit_vec.Select(x => x * 2)
some_64_bit_vec.Select(x => x * 2)
[0] : 0x8
[1] : 0x9e
[2] : 0x34
[3] : 0x308
```
std::string::String is implemented, as well:
```
0:000> dv
hello_world = "Hello, world!"
empty = ""
new = ""
0:000> dx hello_world
hello_world : "Hello, world!" [Type: collections::string::String]
[<Raw View>] [Type: collections::string::String]
[size] : 0xd [Type: unsigned __int64]
[capacity] : 0xd [Type: unsigned __int64]
[0] : 72 'H' [Type: char]
[1] : 101 'e' [Type: char]
...
[12] : 33 '!' [Type: char]
0:000> dx empty
empty : "" [Type: collections::string::String]
[<Raw View>] [Type: collections::string::String]
[size] : 0x0 [Type: unsigned __int64]
[capacity] : 0x0 [Type: unsigned __int64]
```
VecDeque and LinkedList are also implemented.
My biggest concern is the implementation for Option due to the different layouts it can receive based on whether the sentinel value can be embedded with-in the Some value or must be stored separately.
It seems to work, but my testing isn't exhaustive:
```
0:000> dv
three = { Some 3 }
none = { None }
no_str = { None }
some_str = { Some "Hello!" }
0:000> dx three
three : { Some 3 } [Type: core::option::Option<i32>]
[<Raw View>] [Type: core::option::Option<i32>]
[size] : 0x1 [Type: ULONG]
[value] : 3 [Type: int]
[0] : 3 [Type: int]
0:000> dx none
none : { None } [Type: core::option::Option<i32>]
[<Raw View>] [Type: core::option::Option<i32>]
[size] : 0x0 [Type: ULONG]
[value] : 4 [Type: int]
0:000> dx no_str
no_str : { None } [Type: core::option::Option<collections::string::String>]
[<Raw View>] [Type: core::option::Option<collections::string::String>]
[size] : 0x0 [Type: ULONG]
0:000> dx some_str
some_str : { Some "Hello!" } [Type: core::option::Option<collections::string::String>]
[<Raw View>] [Type: core::option::Option<collections::string::String>]
[size] : 0x1 [Type: ULONG]
[value] : 0x4673df710 : "Hello!" [Type: collections::string::String *]
[0] : "Hello!" [Type: collections::string::String]
```
For now all of these visualizations work in windbg, but I've only gotten the visualizations in libcore.natvis working in the VS debugger. My priority is windbg, but somebody else may be interested in investigating the issues related to VS.
You can load these visualizations into a windbg sessions using the .nvload command:
```
0:000> .nvload ..\rust\src\etc\natvis\libcollections.natvis; .nvload ..\rust\src\etc\natvis\libcore.natvis
Successfully loaded visualizers in "..\rust\src\etc\natvis\libcollections.natvis"
Successfully loaded visualizers in "..\rust\src\etc\natvis\libcore.natvis"
```
There are some issues with the symbols that Rust and LLVM conspire to emit into the PDB that inhibit debugging in windbg generally, and by extension make writing visualizations more difficult. Additionally, there are some bugs in windbg itself that complicate or disable some use of the NatVis visualizations for Rust. Significantly, due to NatVis limitations in windbg around allowable type names, you cannot write a visualization for [T] or str. I'll report separate issues as I isolate them.
In the near term, I hope to fill out these NatVis files with more of Rust's core collections and types. In the long run, I hope that we can ship NatVis files with crates and streamline their deployment when debugging Rust programs on windows.
Steve Klabnik