Commit Graph

108 Commits

Author SHA1 Message Date
Josh Soref 5368f3205e
Spelling corrections (#1798)
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
2022-03-30 17:12:18 +08:00
Jermy Li 43b8ae7bff
Bfs optimize vertex-out-out query (#1359)
* not-commit: optimize-query-vertex-out-out-with-in-and
* fix cassandra IN query
* fix batch query missing limit and page info
* add feature supportsQueryWithInCondition()
* improve TableSerializer.writeQueryEdgeCondition()
* impprove flatten IN
* rename BatchQuery to BatchConditionQuery
* fix query from RamTable not flatten
* use BatchMapperIterator to fetch intermediate results(by batch avoid OOM)
* rename to HugeVertexStepByBatch and just enable when needed
* set batch list capacity
* set QUERY_BATCH = 100 for scylladb limit

Change-Id: I1b3c2e942cbfdc9f51cfc43a3ff5aa61b62e22d0
2021-11-03 19:08:59 +08:00
imbajin f22d03fdd6
refact(auth): cache login role in auth client to avoid sending rpc repeatedly (#1507)
* refact(auth): cache login role in auth client to avoid sending rpc repeatedly
* fix: don't create Id every time
* perf: remove useless toLowerCase
* add config for expire time in auth client
* fix StandardAuthManager.inputPassword in auth client
* add option for auth max_capacity
* unified cache_capacity & expired time (also separate invalid cache for pwd)
* add option for token expiration

Co-authored-by: guoshoujing <guoshoujing@baidu.com>
2021-06-25 21:52:40 +08:00
Jermy Li bc9a5a85a4
Add compact api for rocksdb/cassandra/hbase backend (#1378)
* add compact api for rocksdb/cassandra/hbase backend
* increase and standardize some metrics
* increase metrics for cassandra/hbase and unify display format
* fix codecov missing some files coverage by jacococli
* add ScyllaDBMetrics
* make cassandra compact in the scope of keyspace

Change-Id: I26660b28b64089f3b636020e721659db3466267b
2021-06-11 15:57:20 +08:00
zhoney 23e3c31a54
Collection optimize for OLTP algorithm and HugeElement (#1409)
* add Eclipse Collection dependency
* use primitive map for properties of HugeElement
* from Map<Id, HugeProperty<?>> to MutableIntObjectMap<HugeProperty<?>>
* allow config oltp collections(JCF or EC)
* add fastutil choice for collection
* change Set<HugeEdge> of HugeVertex to List<HugeEdge>
* use IntObjectMap and IntIntMap to improve shortest path
* improve paths api by remove redundant sourcesAll and targetsAll
* improve paths api performance by using IntObjectHashMap<IntHashSet>
* extract shortestpath and paths record to hold traversed paths
* implement Int2IntsMap for array paths record
* extract Record and Records for paths and shortest path
* adapt primitive collection for kout/kneighbor
* add objectintmapping unit test
* add UT for CollectionFactory and IdSet

Change-Id: I501a18a70054fad3c40bd6b9e852a82a5c185949
2021-06-09 17:36:47 +08:00
zhoney f6c9850d5c
fix example bug (#1308)
Change-Id: I145e99de88aab45defe6cc00b960d945821edd1e
2020-12-23 22:05:09 +08:00
Jermy Li 26c4498463
fix permission overflow from return results (#969)
* fix permission overflow from return results

also:
 * hide private methods and fields for reflection that require permissions
 * delete stale index of task directly
 
add HugeGraph.java and TaskScheduler.java: separate commits to avoid no git diff

implement: #209
Change-Id: Iccfbd47c213e1e13122190d1242778031e8a957c
2020-04-24 15:42:43 +08:00
Linary 1fe09d0dd2
Increase task result limit (#925)
* Increase task result limit
* Let task input and result property datatype as blob
* Support config task input and result size limit

Change-Id: Ib2c4bdf25e4604d8bea47802e724c0f0934170fb
2020-04-06 19:26:23 +08:00
Jermy Li b38ca33a81
check for unexpected sysprop query (#862)
fix: #861

* workaround: cassandra column with null value can't be queryed with CONTAINS

* string2HugeKey() disallow pass syspprop from gremlin

Change-Id: I319703286f65fd38819b2de91d7c9a18203ceafd
2020-03-30 11:27:57 +08:00
Jermy Li dee579d989
fix task-cancel error with saving failure task (#827)
Change-Id: Ib61c8cdae2e73d346dd611791d47faa30eac6e5a
2020-02-24 09:27:15 +08:00
Linary c19fbea2de Unified license related files name (#733)
Change-Id: I0f9fcc0120c33408765485a8aab7c60b876a27c0
2019-11-05 14:07:42 +08:00
Jermy Li 95eba359e9
support paging for tasks API (#720)
Change-Id: Ie036d433ef74d7c6724bf869425c3fd528f08941
related: #718
2019-11-04 11:26:43 +08:00
imbajin 9d8ceb4dfc Optimize update logic & add a new strategy (#673)
1. if original data is not null but new data is null, use original data instead of null
2. add update strategy "OVERRIDE"
2019-09-24 20:26:07 +08:00
Linary 1c062b8947 Upgrade tinkerpop version to 3.4.3 (#648)
Change-Id: I5443b6a05eea74205a373f093dad27536e04f0be
2019-09-12 19:04:30 +08:00
Jermy Li 26e48a27f1 encode numeric values in secondary index and primary-key (#676)
Change-Id: I6acfad358745a54284f80d236243bfbef61d2deb
2019-09-02 17:15:24 +08:00
Jermy Li 97090e65df fix example1 query error (#638)
Change-Id: I0410c64be5a2ccec2f81a94e6620517a87cf2d7d
2019-08-14 20:43:18 +08:00
Linary 9db6d6a0ef Uniform variable name: vertexes to vertices (#529)
Change-Id: I05c0f52438e3ab74360b396d1c3a278d885a5426
2019-05-27 14:49:40 +08:00
Linary d8d8fe2fa1 Improve some comments (#472)
Change-Id: I237f3e4175eec538aff01c521ec6e93cc8c2fc87
2019-04-19 18:37:05 +08:00
Linary 2890c96e67 Add personalrank and neighborrank RESTful API (#274)
Change-Id: I66e4224b5813aa2d5eadbfe540fb9de203db766b
2019-04-18 22:14:27 -05:00
Jermy Li 727a0e4947 Uniform variable name: Iterator iter (#438)
Change-Id: Iabf708702778a8ecd5eb19bdcf012c4133edff8f
2019-04-09 18:45:17 +08:00
liningrui 0e6359477c Some small modification
Change-Id: I80ff7c0778e52b4ca939f42c7a4e111da5b09406
2019-01-14 15:41:39 +08:00
Zhangmei Li 1885132b5b fix bug: perf example cost time should exclude time of env-init
Change-Id: Iba9964542dc43c8c4d4feb0bb6a2883309e9326a
2019-01-11 17:13:28 +08:00
Zhangmei Li 7164a459ba Support range sortKey feature
implement #271

Change-Id: Ib618c45bc6169a0fb7632dbbf0fb29b7ee0a08e3
2018-12-29 14:15:54 +08:00
Zhangmei Li 1d0e128f88 add the number of pending tasks to metrics
also refactor class name about task to remove prefix 'Huge'

Change-Id: Id21e9d91a0b83b807189fffbee0247219322f234
2018-11-08 21:34:24 +08:00
Zhangmei Li a63d047824 HugeGraph-1322: add support for full-text search
Change-Id: If0a6247d741c346416cc3ff3680460603f727e85
2018-08-09 20:23:26 +08:00
zhangyi51 22c5b7762b hugegraph-889 async implementation of schema deletion and index rebuild
Change-Id: I94e6bb3b5e12097fbe3b7278df625b317137e38b
2018-08-09 20:23:25 +08:00
Zhangmei Li 4676c51614 HugeGraph-1344: support task input parameter
Change-Id: I19b608b6671c50320475e3ff087b2d2fc0e6347a
2018-08-09 20:23:23 +08:00
Zhangmei Li 5a2f8aecb3 HugeGraph-1324: fix "Assertion error: (is_last_reference)"
Change-Id: Ie9f9679f87ac7967de54184191a9e6d29130b32f
2018-08-09 18:17:38 +08:00
Zhangmei Li 57875977ec HugeGraph-387: add async-task framework
Change-Id: I452f3a1c9a7ccf1580075e9fc18456b1b4aae1c6
2018-08-09 18:17:38 +08:00
liningrui cacede83a1 HugeGraph-81: Optimize some performance-related code
Change-Id: Ic35ab56e0648a4e9d1d80ae5d2724839558b871a
2018-08-09 18:17:29 +08:00
Zhangmei Li 3153b077cd HugeGraph-1269: store values in diffrent config files can't be the same
Change-Id: I6233d9f85e7c6a62478a8a73ed14ac2922b99a28
2018-08-09 18:17:26 +08:00
zhangyi51 c51a5d3a85 hugegraph-1234 make limit work when do index query
Change-Id: Ieaba3b5f96a430b41d9fc9dc9d3d6ca2a4315adf
2018-08-09 18:17:23 +08:00
liningrui 0de5469edf HugeGraph-1195: Support eliminate user data on schema
Change-Id: I930dacef54e095d6e283078300916ccb2aa4003a
2018-08-09 18:17:19 +08:00
liningrui 4b15448ed9 HugeGraph-1092: Add Palo backend storage
Change-Id: I5ef5fbe60f5e3988be71d8fc299315cc05420916
2018-08-09 18:17:17 +08:00
Zhangmei Li 6c4e64222e HugeGraph-1165: fix can't parse property with type Date
Change-Id: I2b679143f9ddf23ec05521759a38b51fc9aba8ca
2018-08-09 18:17:09 +08:00
liningrui 55dc0803db HugeGraph-1091: Add mysql backend storage
Change-Id: I0239e7cab51d1e06a9b60881e90ab08f10eebf28
2018-08-09 18:17:07 +08:00
Zhangmei Li b8bb74adc9 HugeGraph-1094: write elem-id twice repeatedly
Change-Id: I2ece5054c0c73f3874aad911d2a356596db95301
2018-08-09 18:17:04 +08:00
Zhangmei Li 768f4f8823 HugeGraph-1071: support rate limit
Change-Id: I28ec8d5e26c531ec4621c484ad345f986b6f8c43
2018-08-09 18:17:03 +08:00
Zhangmei Li eb8638ed7c HugeGraph-1077: support directly writing sst file with rocksdb
Change-Id: Iaec8e55f5b32f3d19e7c833586bd4b8b19bce3c6
2018-08-09 18:17:02 +08:00
Zhangmei Li 0819db1626 HugeGraph-1051: fix count() throws "Too many records"
Change-Id: I4d44fd927d05575c68ec1596b47d9c242d16e75e
2018-08-09 18:16:57 +08:00
Zhangmei Li eee7bc1e4c HugeGraph-254: optimize rocksdb insert perf(updateLabelIndex()->doAppend())
PerfExample1: 14000 -> 16000 edges/s

Change-Id: Ic468c83c00019efcb041f212ded0fe104701b7a5
2018-08-09 18:16:56 +08:00
Zhangmei Li 1a42c6aa87 HugeGraph-1044: add query perf test to PerfExample
Change-Id: I04502a55be236784cc8ebe909dc8f5ccfab1251c
2018-08-09 18:16:55 +08:00
zhangyi51 edcf50776a hugegraph-81 fix bug: config file in jar can't be read
Change-Id: I77f6f6b8f1d66533756659f696915fb1edb22ff9
2018-08-09 18:16:53 +08:00
liningrui 612b7b60f2 HugeGraph-81: Fixed bug that vertex id formatted error in example2
Change-Id: Id43c291f65d3cce46a9338b64a70989b8e9e77c5
2018-08-09 18:16:53 +08:00
Zhangmei Li 6140a35625 HugeGraph-1035: speed up core-test
Change-Id: I5ec9b97cd355da42409452f78149e523a3ceb4da
2018-08-09 18:16:51 +08:00
Zhangmei Li a7786e5cb0 HugeGraph-991: renamae {search}index/query to {range}index/query
Change-Id: I823046317b05e34f8a5a99ebafa6963bdc91104b
2018-08-09 18:16:45 +08:00
liningrui 77c950aa6b HugeGraph-589: Add schema id for all schema element
Change-Id: I5b348acebc35cbd7b864a2a4ad8bb47bd9a5d631
2018-08-09 18:16:44 +08:00
Zhangmei Li b7db9b527a HugeGraph-929: implement RocksDB driver
Change-Id: I9baa4b4e648848eb9f3f017ad0c8c429f07717b7
2018-08-09 18:16:42 +08:00
Zhangmei Li 1c6a1460b8 HugeGraph-918: compatible with tinnkerpop tx
also implement HugeGraph-941

Change-Id: I05fa0c99678ecaddfdd586818a0091801f67ecd7
2018-08-09 18:16:40 +08:00
zhangyi51 d883546a0c hugegraph-895 delete error index when error results queried to reduce error indexes introduced by addVertex override
Change-Id: I878592c3817bfea8280bd1190232cccc6c5d48f9
2018-08-09 18:16:38 +08:00