Commit Graph

7154 Commits

Author SHA1 Message Date
Andrew Noyes bd9faae1e7 Add a unit test that repros #2406 2019-12-04 16:45:32 -08:00
Andrew Noyes 4f943be21d Move DeterministicRandom impl to its own translation unit
This will allow me to recompile faster after making changes, and should
(slightly) speed up overall compilation.

I manually verified that the unseed matched for one test before and
after this change, so I probably didn't screw up the refactor
2019-12-04 16:45:32 -08:00
Evan Tschannen 5a6bc2aa71 increase the priority of cluster controller recruitment to prefer recruitment over sending serverDBInfo 2019-12-04 16:28:41 -08:00
Evan Tschannen 5f1ef53f62 increase the priority at which the cluster controller registers workers to avoid having a saturated cluster controller recruit a master without all available workers 2019-12-04 16:17:41 -08:00
A.J. Beamon e300cd7573
Merge pull request #2400 from atn34/atn34/ubsan-blitz
Fix more UBSAN reports
2019-12-04 13:34:07 -08:00
Andrew Noyes c8e8d0c8ab Fix UBSAN report
/home/anoyes/workspace/foundationdb/fdbclient/SnapshotCache.h:36:9: runtime error: null pointer passed as argument 2, which is declared to never be null
2019-12-04 09:42:35 -08:00
Andrew Noyes e6678573db Fix load of bool which is not 0 or 1 2019-12-04 09:42:35 -08:00
Andrew Noyes cc170fe8d8 Fix UBSAN report
/home/anoyes/workspace/foundationdb/fdbclient/SnapshotCache.h:44:10: runtime error: null pointer passed as argument 2, which is declared to never be null
2019-12-04 09:42:34 -08:00
Andrew Noyes 560c1da805 Fix UBSAN report
/home/anoyes/workspace/foundationdb/fdbserver/VersionedBTree.actor.cpp:1606:10: runtime error: null pointer passed as argument 2, which is declared to never be null
2019-12-04 09:42:34 -08:00
Andrew Noyes 9ef1f4da5c
Update fdbserver/storageserver.actor.cpp
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-12-04 09:21:05 -08:00
Andrew Noyes 41583aa576 Guard unlikely indexCode calculation per Steve 2019-12-03 21:48:30 -08:00
Andrew Noyes 3c1ba46cbe Allow any valid int to be loaded as an Option
We assume that this is kosher in FuzzApiCorrectness, so make it kosher
This fixes some UBSAN reports
2019-12-03 18:07:31 -08:00
Andrew Noyes 854c94c5ad Fix another "binding reference to nullptr" 2019-12-03 17:39:17 -08:00
Andrew Noyes 2aeb9e0cbf Fix UBSAN report 2019-12-03 16:20:39 -08:00
Andrew Noyes 46d10dc7dc Fix "null passed as argument declared not null"
Fix several such reports from ubsan

E.g.

/Users/anoyes/workspace/foundationdb/flow/Arena.h:794:16: runtime error: null pointer passed as argument 1, which is declared to never be null
2019-12-03 14:46:53 -08:00
Andrew Noyes 47de9d9d6e Fix UBSAN report
/Users/anoyes/workspace/foundationdb/fdbserver/TLogInterface.h:149:8: runtime error: load of value 232, which is not a valid value for type 'bool'
    #0 0xc608fb in TLogPeekReply::TLogPeekReply(TLogPeekReply const&) /Users/anoyes/workspace/foundationdb/fdbserver/TLogInterface.h:149
    #1 0x242bf87 in ILogSystem::ServerPeekCursor::ServerPeekCursor(TLogPeekReply const&, LogMessageVersion const&, LogMessageVersion const&, int, int, bool, long, Tag) /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:35
    #2 0x242da77 in ILogSystem::ServerPeekCursor::cloneNoMore() /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:47
    #3 0x24362d5 in ILogSystem::MergedPeekCursor::cloneNoMore() /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:325
    #4 0x244bf45 in ILogSystem::MultiCursor::cloneNoMore() /Users/anoyes/workspace/foundationdb/fdbserver/LogSystemPeekCursor.actor.cpp:838
    #5 0x36b5a36 in a_body1cont5loopBody1 /Users/anoyes/workspace/foundationdb/fdbserver/storageserver.actor.cpp:2621
    #6 0x36b3110 in a_body1cont5loopHead1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8664
    #7 0x36b07fe in a_body1cont5 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8576
    #8 0x36abda8 in a_body1cont4when1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8582
    #9 0x36a8dc2 in a_body1cont4 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8454
    #10 0x36a4bf6 in a_body1cont3break1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8489
    #11 0x36a2c01 in a_body1cont3loopBody1cont1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8505
    #12 0x369fd36 in a_body1cont3loopBody1when1 /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8513
    #13 0x3700dcb in a_callback_fire /Users/anoyes/build/foundationdb/fdbserver/storageserver.actor.g.cpp:8528
    #14 0x36e5210 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    #15 0x4dfb2a in SAV<Void>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    #16 0x2484b07 in a_body1loopBody1cont1 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1526
    #17 0x24822cf in a_body1loopBody1cont2 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1535
    #18 0x247e228 in a_body1loopBody1when1 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1541
    #19 0x249be87 in a_callback_fire /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:1556
    #20 0x249668f in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    #21 0x4dfb2a in SAV<Void>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    #22 0x80557e in a_body1when1 /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11591
    #23 0x8916ef in a_callback_fire /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11620
    #24 0x8735f5 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    #25 0x4dfb2a in SAV<Void>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    #26 0x24820f8 in a_body1cont1loopBody1when1 /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:860
    #27 0x249c852 in a_callback_fire /Users/anoyes/build/foundationdb/fdbserver/LogSystemPeekCursor.actor.g.cpp:886
    #28 0x249786c in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    #29 0xc9d2dc in SAV<TLogPeekReply>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    #30 0x248b39f in a_body1cont2 /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11858
    #31 0x2489d02 in a_body1when1 /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11865
    #32 0x249a150 in a_callback_fire /Users/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11880
    #33 0x2492a4f in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    #34 0xc9d2dc in SAV<TLogPeekReply>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    #35 0x248df9b in a_body1cont2 /Users/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:2762
    #36 0x248b7da in a_body1when1 /Users/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:2769
    #37 0x2499c88 in a_callback_fire /Users/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:2784
    #38 0x2492371 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    #39 0xc9d2dc in SAV<TLogPeekReply>::finishSendAndDelPromiseRef() /Users/anoyes/workspace/foundationdb/flow/flow.h:479
    #40 0xc60fb3 in void SAV<TLogPeekReply>::sendAndDelPromiseRef<TLogPeekReply&>(TLogPeekReply&) /Users/anoyes/workspace/foundationdb/flow/flow.h:472
    #41 0xc1137a in NetSAV<TLogPeekReply>::receive(ArenaObjectReader&) /Users/anoyes/workspace/foundationdb/fdbrpc/fdbrpc.h:111
    #42 0x78eda75 in a_body1cont1 /Users/anoyes/workspace/foundationdb/fdbrpc/FlowTransport.actor.cpp:652
    #43 0x78f7967 in a_body1cont2 /Users/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:2369
    #44 0x78ed4d8 in a_body1when1 /Users/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:2375
    #45 0x791af45 in a_callback_fire /Users/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:2390
    #46 0x7914670 in fire /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    #47 0x4df0dc in void SAV<Void>::send<Void>(Void&&) /Users/anoyes/workspace/foundationdb/flow/flow.h:447
    #48 0x959891 in void Promise<Void>::send<Void>(Void&&) const /Users/anoyes/workspace/foundationdb/flow/flow.h:778
    #49 0x7b4b022 in Sim2::execTask(Sim2::Task&) (/Users/anoyes/build/foundationdb/bin/fdbserver+0x7b4b022)
    #50 0x7bf9172 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_body1loopBody1cont1(Void const&, int) /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:979
    #51 0x7be7b72 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_body1loopBody1when1(Void const&, int) /Users/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5391
    #52 0x7c32a09 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_callback_fire(ActorCallback<Sim2::RunLoopActor, 0, Void>*, Void) /Users/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5406
    #53 0x7c1fc7d in ActorCallback<Sim2::RunLoopActor, 0, Void>::fire(Void const&) /Users/anoyes/workspace/foundationdb/flow/flow.h:998
    #54 0x4df0dc in void SAV<Void>::send<Void>(Void&&) /Users/anoyes/workspace/foundationdb/flow/flow.h:447
    #55 0x959891 in void Promise<Void>::send<Void>(Void&&) const /Users/anoyes/workspace/foundationdb/flow/flow.h:778
    #56 0x7fe74ae in N2::PromiseTask::operator()() /Users/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:481
    #57 0x7fb7001 in N2::Net2::run() /Users/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:657
    #58 0x7b71bdd in Sim2::_runActorState<Sim2::_runActor>::a_body1(int) /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:989
    #59 0x7b2ee5b in Sim2::_runActor::_runActor(Sim2* const&) /Users/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5608
    #60 0x7b2f272 in Sim2::_run(Sim2* const&) /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:987
    #61 0x7b2f2d2 in Sim2::run() /Users/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:996
    #62 0x2104064 in main /Users/anoyes/workspace/foundationdb/fdbserver/fdbserver.actor.cpp:1793
    #63 0x7fb7c6561504 in __libc_start_main (/lib64/libc.so.6+0x22504)
    #64 0x464914  (/Users/anoyes/build/foundationdb/bin/fdbserver+0x464914)
2019-12-03 12:51:36 -08:00
Andrew Noyes 6bde67f2b3 Fix UBSAN report
/home/anoyes/workspace/foundationdb/fdbserver/Ratekeeper.actor.cpp:86:8: runtime error: load of value 1231493777, which is not a valid value for type 'limitReason_t'
    #0 0x310e961 in StorageQueueInfo::StorageQueueInfo(StorageQueueInfo&&) /home/anoyes/workspace/foundationdb/fdbserver/Ratekeeper.actor.cpp:86
    #1 0x310eacd in MapPair<UID, StorageQueueInfo>::MapPair<UID, StorageQueueInfo>(UID&&, StorageQueueInfo&&) /home/anoyes/workspace/foundationdb/flow/IndexedSet.h:242
    #2 0x310b35e in MapPair<std::decay<UID>::type, std::decay<StorageQueueInfo>::type> mapPair<UID, StorageQueueInfo>(UID&&, StorageQueueInfo&&) /home/anoyes/workspace/foundationdb/flow/IndexedSet.h:258
    #3 0x30a8b79 in a_body1 /home/anoyes/workspace/foundationdb/fdbserver/Ratekeeper.actor.cpp:195
    #4 0x309b529 in TrackStorageServerQueueInfoActor /home/anoyes/build/foundationdb/fdbserver/Ratekeeper.actor.g.cpp:495
    #5 0x309b9be in trackStorageServerQueueInfo(RatekeeperData* const&, StorageServerInterface const&) /home/anoyes/workspace/foundationdb/fdbserver/Ratekeeper.actor.cpp:194
    #6 0x30cff63 in a_body1loopBody1when1cont1 /home/anoyes/workspace/foundationdb/fdbserver/Ratekeeper.actor.cpp:303
    #7 0x30cd9da in a_body1loopBody1when1when1 /home/anoyes/build/foundationdb/fdbserver/Ratekeeper.actor.g.cpp:1170
    #8 0x30ed4dd in a_callback_fire /home/anoyes/build/foundationdb/fdbserver/Ratekeeper.actor.g.cpp:1185
    #9 0x30e6d81 in fire /home/anoyes/workspace/foundationdb/flow/flow.h:998
    #10 0x4df0dc in void SAV<Void>::send<Void>(Void&&) /home/anoyes/workspace/foundationdb/flow/flow.h:447
    #11 0x959891 in void Promise<Void>::send<Void>(Void&&) const /home/anoyes/workspace/foundationdb/flow/flow.h:778
    #12 0x7b4b018 in Sim2::execTask(Sim2::Task&) (/home/anoyes/build/foundationdb/bin/fdbserver+0x7b4b018)
    #13 0x7bf9168 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_body1loopBody1cont1(Void const&, int) /home/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:979
    #14 0x7be7b68 in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_body1loopBody1when1(Void const&, int) /home/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5391
    #15 0x7c329ff in Sim2::RunLoopActorState<Sim2::RunLoopActor>::a_callback_fire(ActorCallback<Sim2::RunLoopActor, 0, Void>*, Void) /home/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5406
    #16 0x7c1fc73 in ActorCallback<Sim2::RunLoopActor, 0, Void>::fire(Void const&) /home/anoyes/workspace/foundationdb/flow/flow.h:998
    #17 0x4df0dc in void SAV<Void>::send<Void>(Void&&) /home/anoyes/workspace/foundationdb/flow/flow.h:447
    #18 0x959891 in void Promise<Void>::send<Void>(Void&&) const /home/anoyes/workspace/foundationdb/flow/flow.h:778
    #19 0x7fe74a4 in N2::PromiseTask::operator()() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:481
    #20 0x7fb6ff7 in N2::Net2::run() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:657
    #21 0x7b71bd3 in Sim2::_runActorState<Sim2::_runActor>::a_body1(int) /home/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:989
    #22 0x7b2ee51 in Sim2::_runActor::_runActor(Sim2* const&) /home/anoyes/build/foundationdb/fdbrpc/sim2.actor.g.cpp:5608
    #23 0x7b2f268 in Sim2::_run(Sim2* const&) /home/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:987
    #24 0x7b2f2c8 in Sim2::run() /home/anoyes/workspace/foundationdb/fdbrpc/sim2.actor.cpp:996
    #25 0x21040a6 in main /home/anoyes/workspace/foundationdb/fdbserver/fdbserver.actor.cpp:1793
    #26 0x7f03492ba504 in __libc_start_main (/lib64/libc.so.6+0x22504)
    #27 0x464914  (/home/anoyes/build/foundationdb/bin/fdbserver+0x464914)
2019-12-03 12:49:12 -08:00
Andrew Noyes 1721658210 Ignore -fsanitize=alignment for UBSAN
Until we fix all occurrences
2019-12-02 16:20:06 -08:00
Andrew Noyes 4022ca1381 Add USE_UBSAN cmake option 2019-12-02 16:20:06 -08:00
Andrew Noyes 7f263a2614 Require callers to alignment requirements for aligned_alloc 2019-12-02 15:30:54 -08:00
Andrew Noyes ff8758b1fd Request alignment that's at least sizeof(void*)
According to https://en.cppreference.com/w/c/memory/aligned_alloc#Notes,
aligned_alloc may return nullptr if it doesn't like the requested
alignment. Let's also detect if nullptr is returned.
2019-12-02 12:51:33 -08:00
Andrew Noyes b086dbecac Fix another UBSAN error
fdbserver/sqlite/sqlite3.amalgamation.c:14709:15: runtime error: left shift of 205 by 24 places cannot be represented in type 'int'
2019-12-02 12:51:33 -08:00
Andrew Noyes 36e9f40fc2 Fix negative shift exponent
fdbserver/KeyValueStoreSQLite.actor.cpp:438:11: runtime error: shift exponent -1 is negative
2019-12-02 12:51:33 -08:00
Andrew Noyes e0bf7c4d65 Fix signed integer overflow
Not sure if this is the right fix or not

fdbserver/Ratekeeper.actor.cpp:557:40: runtime error: signed integer overflow: -9223372036854775808 - 9223372036854775807 cannot be represented in type 'long long'
2019-12-02 12:51:33 -08:00
Meng Xu a2c84d932d
Merge pull request #2396 from atn34/atn34/fix-invalid-vptr
Fix invalid vptr
2019-11-27 21:16:49 -08:00
Meng Xu 7eaf76bacf
Merge pull request #2389 from atn34/atn34/default-init-flatbuffers
Default initialize absent flatbuffers members
2019-11-27 21:16:27 -08:00
Meng Xu e49e87fae1
Merge pull request #2394 from atn34/atn34/fix-ref-binding-to-null
Fix reference binding to nullptr
2019-11-27 17:33:45 -08:00
Andrew Noyes 8fc74e3182 Fix UBSAN error
Since QuorumCallback<T> is a non trivial type, we need to construct it
before we interact with it

This change fixes the following UBSAN message
/Users/anoyes/workspace/foundationdb/flow/genericactors.actor.h:930:18: runtime error: member access within address 0x0001243f63d0 which does not point to an object of type 'Callback<Standalone<StringRef> >'
0x0001243f63d0: note: object has invalid vptr
2019-11-27 13:16:48 -08:00
Meng Xu 70dcbeb204
Merge pull request #2390 from atn34/atn34/fix-potential-ub
Fix a potential UB instance
2019-11-26 23:25:57 -08:00
Andrew Noyes f320f6c174 Fix occurrence of undefined behavior
UBSAN has this to say:

flow/Arena.h:982:10: runtime error: reference binding to null pointer of type 'KeyValueRef'

After this change UBSAN no longer complains about this occurrence
2019-11-26 21:34:24 -08:00
Andrew Noyes c4e01301b0 Fix a potential UB instance
Writing a value which is not 0 or 1 to the underlying memory of a bool
is undefined behavior. Conformant flatbuffers implementations must
accept bytes that are not 0 or 1 as booleans [1]. (Conformant
implementations are only allowed to write the byte 0 or 1 as a boolean
[1])

So this protects us from undefined behavior if we ever read a
flatbuffers message written by an almost-conformant implementation.

[1]: https://github.com/dvidelabs/flatcc/blob/master/doc/binary-format.md#boolean
2019-11-26 11:18:17 -08:00
Andrew Noyes 17ab2f8e00 Default initialize absent flatbuffers members 2019-11-26 10:58:29 -08:00
Evan Tschannen 8bc5574836
Merge pull request #2385 from etschannen/post-release-cleanup-6.2.11
Post release cleanup 6.2.11
2019-11-22 15:34:12 -08:00
Evan Tschannen 74500161f4 update CMake version 2019-11-22 15:30:10 -08:00
Evan Tschannen 54a22646ac update installer WIX GUID following release 2019-11-22 15:28:36 -08:00
Evan Tschannen 2fecc4ad0d update versions target to 6.2.12 2019-11-22 15:28:36 -08:00
Evan Tschannen 66ac949e6a
Merge pull request #2380 from etschannen/release-6.2
Team trackers for bad teams that contain a removed servers must be cancelled
2019-11-22 11:11:49 -08:00
Evan Tschannen 593db826b2 removed accidental merge code 2019-11-22 11:04:35 -08:00
Evan Tschannen 71f63cd026
Merge pull request #2381 from etschannen/prepare-release-6.2.11
update installer WIX GUID following release
2019-11-22 11:03:30 -08:00
Evan Tschannen 3bd203b4f3 update installer WIX GUID following release 2019-11-22 11:02:54 -08:00
Evan Tschannen 44765a46ac Merge branch 'release-6.2' of github.com:apple/foundationdb into release-6.2
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2019-11-22 11:00:22 -08:00
Evan Tschannen dd3b30a5b5
Merge pull request #2371 from ajbeamon/fix-latency-band-deconfiguration
Fix cases where latency band config could be discarded during recovery or process start
2019-11-22 10:59:29 -08:00
Evan Tschannen 837f852ad9 updated documentation for 6.2.11 2019-11-22 10:58:17 -08:00
Evan Tschannen 3a3ab5664b fix: team trackers for bad teams that contain a removed servers must be cancelled or the cluster will falsely report those teams as failed 2019-11-22 10:20:13 -08:00
Evan Tschannen b664eba7d7
Merge pull request #2377 from etschannen/release-6.2
Errors from net()->connect on clients did get reflected in the failure monitor
2019-11-22 10:13:52 -08:00
Evan Tschannen 746b357b7f fix: simulation should not allow connections to dead processes 2019-11-21 20:36:40 -08:00
Evan Tschannen 27cb299d84 simulation can sometimes randomly hang or throw connection_failed, instead of always doing one or the other 2019-11-21 16:24:18 -08:00
Evan Tschannen 067dc55bfb fix: making _conn a state variable was keeping connections open that should be closed 2019-11-21 16:08:32 -08:00
Evan Tschannen 569c6d4476 throws of connection_failed() from net()->connect did not result in clients marking a connection as failed in the failure monitor 2019-11-21 13:08:59 -08:00
Evan Tschannen 2727b91c46 simulation tests network connections failing due to errors instead of just hanging 2019-11-21 12:33:07 -08:00