Commit Graph

3281 Commits

Author SHA1 Message Date
A.J. Beamon eab468fecc Remove extra line caused by commit issue 2021-04-14 09:32:48 -07:00
A.J. Beamon feede1d2f6 Fix line length of test macro + comments to be within the 120 character limit 2021-04-13 10:48:52 -07:00
RenxuanW c8b27e71c5 Revert TraceEvent
We've found the problem (issue #4640), so we no longer need the TraceEvent.
2021-04-12 15:03:47 -07:00
RenxuanW dc00d99626 Log FileBackupLogRangeStart before calling getLogRanges() in .
It will tell us if or why this function is legitimately trying to use too much ram. getLogRange() should normally return about 20 items in the result. If the inputs are trash, it could return far more.

If it isn’t the case, then there’s something else wrong that has corrupted something such that when we try to allocate memory.
2021-04-09 15:02:45 -07:00
RenxuanW 551dfa6ad8 Move description of RESTORE_IGNORE_LOG_FILES to header file. 2021-04-07 10:21:24 -07:00
RenxuanW fadc9cccee Use knob RESTORE_IGNORE_LOG_FILES in restore.
Rename IGNORE_LOG_FILES to RESTORE_IGNORE_LOG_FILES. Also, this knob should be used in regular restore, not parallel restore.
2021-04-07 10:05:56 -07:00
RenxuanW 1b6ad42db8 Use a knob to completely ignore log files 2021-04-06 19:07:01 -07:00
Vishesh Yadav 91afa3cc9c
Merge pull request #4515 from halfprice/zhewu/update-fault-tolerance-calculation
Populate min_replicas_remaining fields in the cluster status for all regions. This gives better picture of the data replicas currently exist in the database.
2021-03-31 13:06:17 -07:00
Steve Atherton 7d44af3fec
Merge pull request #4568 from jzhou77/master
Use the restored range in the actual restore
2021-03-30 09:58:31 -07:00
Jingyu Zhou 4c73e838ab Fix a comment on backup log data location 2021-03-30 09:26:48 -07:00
Zhe Wu 2648d19edd Update comment 2021-03-29 15:10:57 -07:00
Jingyu Zhou 2b4744ad8a Reword comments. 2021-03-29 13:43:43 -07:00
Jingyu Zhou 55ef40864f Add comments for touched functions 2021-03-29 10:09:07 -07:00
Jingyu Zhou 1a0ef39e5d Fix keyRangesFilter with logsOnly flag
Remove the Sev 40 trace event.
2021-03-27 21:48:27 -07:00
Zhe Wu c3aff4340f Address merge conflicts 2021-03-26 11:36:02 -07:00
Jingyu Zhou a9d8806828 Use the restored range in the actual restore 2021-03-25 10:07:25 -07:00
Jingyu Zhou 3fefebb23a
Merge pull request #4554 from sfc-gh-satherton/restore-log-filtering2
Mutation log filtering during restore
2021-03-24 22:54:09 -07:00
Steve Atherton 720c0b3a4f
Added const.
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2021-03-24 14:36:55 -07:00
A.J. Beamon ed811008ca
Merge pull request #4504 from sfc-gh-nwijetunga/network_busyness
Monitor Network Thread Busyness
2021-03-24 09:44:09 -07:00
Steve Atherton a6136ffc62 Applied clang-format to recent changes. 2021-03-24 06:51:08 -07:00
Steve Atherton 17a19021fb Added mutation batch filtering during restore, to the extent possible where each restore mutation log task only sees one block of data (usually 1MB) which can contain incomplete and therefore unfilterable mutation sets. 2021-03-24 06:38:50 -07:00
Nim Wijetunga de1c354b12 address pr comments 2021-03-23 20:56:37 +00:00
Andrew Noyes 7f4bde74ef Add some documentation 2021-03-18 23:34:44 +00:00
Nim Wijetunga 73571ad52e Merge branch 'master' of github.com:sfc-gh-nwijetunga/foundationdb into network_busyness
* 'master' of github.com:sfc-gh-nwijetunga/foundationdb: (55 commits)
  Port lost changes from #4004
  Fix markdown
  Factor our refreshTransaction gadget
  Align FastAllocator memory to 4096 for size >= 4096
  Fix heap-use-after-free's
  Disable lsan in joshua
  Manually apply changes made to BackupContainer.actor.cpp on release-6.3 to the new files where that code is located.
  add release branch instructions
  Fix PR number in release note link
  fixed a simulation bug where a process on an unreliable machine would be considered reliable by the simulator
  Add tini as PID 1 to docker image (#4363)
  Fix typo
  Fix broken_promise bug
  Explain FastAllocator<4096> special case
  Inform LSAN that we have unaligned pointers
  apply clang-format to flow/Platform.h
  Fix typos
  Address review comments
  Change the macro that forbids exit() calls to be a static assertion
  Add versionstamp serialization information
  ...
2021-03-18 18:58:43 +00:00
Andrew Noyes 0919eff187 Fix data race in fdb_get_server_protocol 2021-03-18 18:24:12 +00:00
Zhe Wu 27060cf4a9 Populate min_replicas_remaining field for all regions during cluster status computation. This provides more information about the number of available replicas currently in the database. 2021-03-17 15:45:12 -07:00
Nim Wijetunga ea922aa648 address pr comments 2021-03-17 18:16:18 +00:00
Nim Wijetunga 822cbf895d address pr comments 2021-03-16 23:29:02 +00:00
Nim Wijetunga 0c6b9bc541 remove guard around g_network 2021-03-16 21:53:24 +00:00
Nim Wijetunga bf5f83d2bf address pr comments 2021-03-16 21:49:36 +00:00
Nim Wijetunga 5f079904d4 add unit test 2021-03-16 18:30:39 +00:00
Nim Wijetunga 17f9da6962 remove traces 2021-03-16 16:47:50 +00:00
A.J. Beamon 3751ecd11f
Merge pull request #4503 from sfc-gh-ajbeamon/merge-release-6.3-into-master
Final merge of release-6.3 into master
2021-03-16 09:29:05 -07:00
Trevor Clinkenbeard ffb8e27f43
Merge pull request #4461 from sfc-gh-anoyes/anoyes/fix-thread-leak
Fix several leaks
2021-03-15 22:03:45 -07:00
Andrew Noyes 191230a7b3 Factor our refreshTransaction gadget 2021-03-16 00:51:20 +00:00
A.J. Beamon a892eec57d Manually apply changes made to BackupContainer.actor.cpp on release-6.3 to the new files where that code is located. 2021-03-15 16:41:22 -07:00
A.J. Beamon 25c4880ebe Merge branch 'release-6.3' into merge-release-6.3-into-master (temporarily discard all changes to BackupContainer.actor.cpp)
# Conflicts:
#	fdbclient/BackupContainer.actor.cpp
#	fdbserver/Knobs.h
2021-03-15 16:41:04 -07:00
Nim Wijetunga f0d79b3d86 Inital implementation of network busyness 2021-03-15 23:23:56 +00:00
Trevor Clinkenbeard 766b9d24d6
Merge branch 'master' into remove-invalid-backup-status-fields 2021-03-14 18:57:03 -07:00
Andrew Noyes d45481decc Fix broken_promise bug 2021-03-13 01:16:27 +00:00
Andrew Noyes 6cacf3e002 Fix potential heap-use-after-free
If we destroy tr here then we might not be able to dereference |self|.
Keep the window where |self| might have been deleted small.
2021-03-12 00:26:26 +00:00
Andrew Noyes a76c7b9754 Fix several memory leaks and a thread leak 2021-03-11 23:58:32 +00:00
A.J. Beamon 0dd9fdf9de Throw an error if the local client is disabled and there are no external clients 2021-03-11 14:09:52 -08:00
Vishesh Yadav 2c316db94d Merge remote-tracking branch 'apple/master' into master-format-final
Also format the changes
2021-03-11 12:54:26 -08:00
Andrew Noyes be980ebb55
Merge pull request #4354 from sfc-gh-clin/add-advanceversion
Add advanceversion into special keys
2021-03-11 11:43:53 -08:00
Russell Sears d17702e7c4
Merge pull request #4463 from sfc-gh-ajbeamon/mvc-dont-copy-with-single-thread
Multi-version client: don't copy external libraries if we aren't using multiple threads
2021-03-11 11:16:54 -08:00
Vishesh Yadav d7252da951 clang-format: Fix the TEST() macros which require comments in line 2021-03-10 16:50:53 -08:00
Vishesh Yadav 2cd3f45fd6 Merge remote-tracking branch 'apple/release-6.3' into master-format-final
release-6.3 was recently merged, and there were two PRs which were
merged in between and got those changes in here. Hence, since all the
changes were in, discarded the incoming changes and accepted all
current.
2021-03-10 16:50:44 -08:00
A.J. Beamon fe4ff420a3 Don't copy the external libraries if you aren't using more than one thread 2021-03-10 15:09:30 -08:00
Markus Pilman cfbaaa6001 Merge remote-tracking branch 'origin/release-6.2' into release-6.3 2021-03-10 14:30:47 -07:00