Commit Graph

1496 Commits

Author SHA1 Message Date
Jade Peng 020b7dc00a
refactor(core): improve code by flow idea suggests (#2000)
Co-authored-by: jadepeng <jqpeng@iflytek.com>
2022-11-11 23:34:00 +08:00
imbajin b1b12098fe adapt the latest version & clean code
also fix the sec alert
2022-11-09 18:24:54 +08:00
Jermy Li 75e004a4e3 fix lpa not exist c_label property when a request pass source_label (#78)
Change-Id: Ib2a9766e29708fa4fcbaec152f5c9e571c96a0a6
2022-11-09 18:24:54 +08:00
zhoney c3e9f8ec06 fix fusiform and kcore min_groups args default value (#79)
Change-Id: I843c595172d7b45b894d764bf859dec61a35c8b5
2022-11-09 18:24:54 +08:00
zhoney b4fa2a16ac fix ap compile error for edition 0.11 (#72)
Change-Id: Ibf0f415ee6bd9cf0ff8b598e2166cc70667c49b7
2022-11-09 18:24:54 +08:00
houzhizhen f63f056034 performance improvement: the meetNode is invoked only when node.distance is parent.distance() + 1 (#69) 2022-11-09 18:24:54 +08:00
houzhizhen 476477bb7b add StressCentrality v2 (#65)
* add StressCentralityAlgorithmV2
* add BfsTraverser and ClosenessCentralityAlgorithmV2
2022-11-09 18:24:54 +08:00
Jermy Li 6c5e204c38 add depth-first version betweennes centrality (#68)
Change-Id: Id4fc070268d8957ef2764bffbb3c762d9c034a0d
2022-11-09 18:24:54 +08:00
Jermy Li f1ad2e5623 fix closeness distance: 1 extra length is calculated (#67)
Change-Id: I35bc20698bf93002a530b95688cd0359a9154fdf
2022-11-09 18:24:54 +08:00
houzhizhen 708ace7deb add betweenness algorithm (#63)
* add betweenness BetweennessCentralityAlgorithmV2
2022-11-09 18:24:54 +08:00
houzhizhen 71f6e81796 update Betweenness with Stressness (#60) 2022-11-09 18:24:54 +08:00
Jermy Li 9926fc6e05 add export_community for lounvain (#43)
Change-Id: I01e402fc99669f53544279c752f81d886c6ce28f
2022-11-09 18:24:54 +08:00
Jermy Li 4caf261216 add with_boundary parameter for betweeness (#42)
* improve betweeness by remove boundary vertex of path

Change-Id: I76924daf8d9da113ab7a1aeac536c6080eccb296

* add with_boundary parameter for betweeness

also fix lpa count comms with limit

Change-Id: Iaf675cd87a8dc0b5ef75476144bc8141f2dd4385
2022-11-09 18:24:54 +08:00
Jermy Li 37bd251708 fix DegreeCentrality degree count limit OOL (#39)
Change-Id: I1b055130c75edce039a97822d9154e1214bc80c2
2022-11-09 18:24:54 +08:00
Jermy Li 148d6250dc add skipi_isolated param for louvain (#37)
Change-Id: Ib24ede9c20bb2c23a3f06fe72c53be2342295fd4
2022-11-09 18:24:54 +08:00
houzhizhen eb6398c4b7 add parameter top to print the top result in job result (#32)
* add parameter top to print the top result in job result
2022-11-09 18:24:54 +08:00
Jermy Li 7ef0305da7 fix server id/role NPE in initServerInfo when truncate temp graph (#28)
Change-Id: Iedbafa9a31ed3f5b3fb35ccb0c583ec0a0cfc6ac
2022-11-09 18:24:54 +08:00
Jermy Li 353d90698f fix no auth with worker thread of olap algo (#27)
call graph close instead of closeTx

Change-Id: I0e329280b067f34daec69c9b1b2b81a6cd3309bf
2022-11-09 18:24:54 +08:00
Zhangmei Li 0d6b871aba rebase master(0.11.1)
Change-Id: I565602945a26c2a575baaa3f17d084b65399a009
2022-11-09 18:24:54 +08:00
Jermy Li 53025bd43b add BOTH direction support for triangle_count/cluster_coeffcient (#24)
change log:
1. add BOTH direction support for triangle_count/cluster_coeffcientith.
2. fix extra triangle count with multi edges between tow adjacent vertices.
3. set default value of direction to BOTH for degree_centrality and cluster_coeffcient .
4. add workers for triangle_count and cluster_coeffcient.
5. fix closeness: multi shortest paths cause results illogical.
6. rename rings_detect to rings, rename limit to each_limit which means limit number of rings of each source vertex, and don't do dedup if passed each_limit > 0.
7. unify top for 4 centrality algos: sorted results when top = -1, unsorted results when top = 0.
8. fusiform: rename top to top_similars (expected >= 0).
9. fusiform/rings: add limit param which means max number of results, and remove capacity param and hardcode to 100000000.

Change-Id: I9ddf8553e6d86b99adbff8b972890d69d623fa1a
2022-11-09 18:24:54 +08:00
Jermy Li 6ef3c0e291 add subgraph_stat algorithm (#23)
Change-Id: If4faafc1a952186c17314c599f611f7ab6132b7b
2022-11-09 18:24:54 +08:00
houzhizhen 8385db2177 add weak connected component analysis (#21) 2022-11-09 18:24:54 +08:00
houzhizhen 31ae32b67a add page rank algorithm (#19) 2022-11-09 18:24:54 +08:00
Jermy Li 71109cffde fix algorithm can't stop caused by threads exception (#18)
Change-Id: I546682b19fb5a84a65dc2a3bd77d62b386722bfa
2022-11-09 18:24:54 +08:00
zhoney 464d99080f Support ring detect count (#17)
* support rings count for rings-detect ap algo
* fix direction can't be null or Both for triangle and clusterCoeffcient
* fix source_clabel error for rings-detect, fusiform and kcore
* change limit meaning for fusiform similarity

Change-Id: I5a4ccbf46b47c13ea2eafe7f3e335dc6aea4a83c
2022-11-09 18:24:54 +08:00
Jermy Li beda055142 fix parallel LPA not commit by threads (#16)
Change-Id: I8eaaeccaa0b23048a9d0f597080186c069b9799b
2022-11-09 18:24:54 +08:00
Jermy Li 4763e876bc support parallel: Louvain,LPA,Rings,K-Core,Fusiform (#15)
* optimize louvain by multi threads
* implement louvain threads
* fix race condition
* implement merge community by multi threads
* remove debug info
* fix genId race condition
* compatible with serial and parallel computing
* support parallel lpa
* support parallel: Louvain,LPA,Rings,K-Core,Fusiform

Change-Id: I2425d1da58581ea7a61dce72a88355ae3d2dd610
2022-11-09 18:24:54 +08:00
Jermy Li 9c9c26c3cb louvain: add modularity parameter and fix isolated community lost (#14)
* add modularity parameter for louvain
* fix: louvain lost isolated community from one to next pass

Change-Id: I6a7dadc80635429aa2898939aa337aae01bc8d12
2022-11-09 18:24:54 +08:00
Jermy Li bc9822d7bb add direction and label parameter for centrality algorithms (#13)
Change-Id: I20b72ea0da673359e2bd21888010290efca81441
2022-11-09 18:24:54 +08:00
Jermy Li 7a72d7df60 optimize filterNonShortestPath() for betweeness_centrality (#12)
network10000 dataset test:
                   before  after
(depth=4 sample=1) 395s    25s
(depth=3 sample=2) 4300s   35s

same as the closeness_centrality

Change-Id: Ia0c557434bf25f9d13a0b1dc19f66024e08c89df
2022-11-09 18:24:54 +08:00
Jermy Li fa3ace6865 fix inconsistent error messages with clabel (#10)
also improve error message cause reason

Change-Id: I15ea8dd651e01ff678a32f19efd3584cd20ffc10
2022-11-09 18:24:54 +08:00
Jermy Li c96d4e94e1 set louvain max community members limit 10w (#9)
Change-Id: I5cc3be846ff4536ebfba1f9bf54a0adda7409036
2022-11-09 18:24:54 +08:00
zhoney c89b0690dd add fusiform_similarity,rings_detect and kcore ap algorithm (#5)
* improve
* move c_label to lower layer and add appendRow(value)
* add community limit 100w for louvain
* improve louvain log
* fix louvain bug

Change-Id: I886ac3e7a3f0dfd49e66fdf544f97f6f7db615df
2022-11-09 18:24:54 +08:00
Jermy Li 9e4e33f41a implement 8 olap algorithms (#4)
* add olap algo api
* improve source filter
* fix louvain shaking with limit degree
* catch exception for lpa and louvain
* add 3 params for lpa: label,source_label,percision
* improve louvain node store
* remove vertices from class Community
* move showCommunity to AbstractCommAlgorithm
* add some parameters to AbstractAlgorithm
* improve louvain log
* improve clearPass and communities check
* split louvain cache
* fix degreeCentrality bug: degree is always < 500

Change-Id: I2341b981dab44f43ac50ae0f8fa5e51b7acc1b5a
2022-11-09 18:24:54 +08:00
imbajin 8869c9ea4a
chore: enable rebase merge (#2003) 2022-11-09 14:56:20 +08:00
青年 0dfb124314
add flatten plugin (#2004) 2022-11-08 22:13:53 +08:00
Simon Cheung 89ebbfdffb
Add thrid-party dependency licenses (#2008)
Co-authored-by: imbajin <jin@apache.org>
2022-11-08 21:14:48 +08:00
Simon Cheung 2fc8869347
check dependencies (#2006) 2022-11-07 21:30:39 +08:00
Jade Peng 09619597d6
refactor: improve rocksdb code (#2002)
Co-authored-by: jadepeng <jqpeng@iflytek.com>
2022-11-03 23:47:32 +08:00
zyxxoo 19ef0ac203
remove word dependency to remove GPL license (#1998) 2022-10-31 12:05:22 +08:00
zyxxoo 66b167422f
fix: User-controlled data in numeric cast [#1987] (#1996)
* fix: User-controlled data in numeric cast [#1987]
2022-10-30 23:29:50 +08:00
imbajin c1d32a8bd0
chore: update ci version & enable it (#1892)
* chore: enable required ci

* Update .asf.yaml

* Update .asf.yaml

* update ci version

* Update .asf.yaml

* Update .asf.yaml

* Update codeql-analysis.yml

* Update .asf.yaml
2022-10-30 22:23:33 +08:00
seagle cc80d27745
fix redundant Checkstyle (#1896)
* add redundant

* add redundant check

* merge line

* update alignment

* delete duplicates check MissingSwitchDefault

* update wrap line

* update code style

* merge dev

* merge master
2022-10-30 16:16:21 +08:00
imbajin 3a22263621
chore: improve gitignore file (#1994) 2022-10-30 16:14:44 +08:00
Jermy Li 1fbd5ffcb5
fix TraversalStrategyProxy() compile error on java 11 (#1982)
compile error on java 11:
  Cannot infer type arguments for new TraversalStrategyProxy<>(strategy)

Change-Id: I6a31cb24d4f37cc6af9ae9769a986593301ea1ff
2022-10-28 00:00:22 +08:00
Jade Peng 07810249a5
refact(core): update dependencies version & adopt the new analyzer (#1989)
jraft 1.3.9->1.3.11
ohc-core 0.7.0->0.7.4
org.apdplat.word 1.3->1.3.1
hanlp portable-1.5.0->portable-1.8.3
lucene-analyzers-smartcn,lucene-core 7.4.0->8.11.2
jcseg-core->2.2.0 ->2.6.2
lz4-java 1.7.1->1.8.0
eclipse-collections 10.4.0->11.1.0
fastutil 8.1.0->8.4.0
jjwt 0.11.2->0.11.5

Co-authored-by: jadepeng <jqpeng@iflytek.com>
2022-10-26 23:11:30 +08:00
Jade Peng 88a8d782b8
improve analyzer code (#1990)
Co-authored-by: jadepeng <jqpeng@iflytek.com>
2022-10-26 22:57:41 +08:00
青年 8abf3d5524
chore: support check license header with RAT & fill the gaps (#1988) 2022-10-26 19:43:41 +08:00
Jade Peng 9b5950e4be
fix(api): fix remoting rpc server not started (#1975)
Co-authored-by: jadepeng <jqpeng@iflytek.com>
2022-10-21 21:40:28 +08:00
dependabot[bot] 99d4481e94
build(deps): bump protobuf-java from 3.21.2 to 3.21.7 (#1973)
Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.2 to 3.21.7.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](https://github.com/protocolbuffers/protobuf/compare/v3.21.2...v3.21.7)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-13 23:08:16 +08:00