Compare commits

..

140 Commits

Author SHA1 Message Date
V_Galaxy e1bd70831a
feat(dist): support docker deployment for PD and Store (#2529)
Co-authored-by: imbajin <jin@apache.org>
2024-05-18 23:18:26 +08:00
Hongjun Li 4c6eda8409 chore: config *.md max length (#2525) 2024-04-20 16:14:34 +08:00
Emmanuel Ferdman 4ba5c9dedd fix: update resource references (#2522) 2024-04-20 16:14:30 +08:00
Liu Xiao a534417dd9 chore: make IDEA support IssueNavigationLink and add icon (#2521) 2024-04-20 16:14:27 +08:00
V_Galaxy 9affa6ecd2 fix(server): avoid overriding backend config in gremlin example script (#2519) 2024-04-20 16:14:22 +08:00
V_Galaxy 1d47fd38e2 fix(server): switch rocksdb backend to memory when executing gremlin example (#2518) 2024-04-20 16:14:19 +08:00
imbajin 3ea3fa6623 fix: maven rat plugin exist partial failure
fix: maven rat plugin exist partial failure
2024-04-20 16:14:10 +08:00
V_Galaxy 4d11dd5e05
fix(pd-store): reset previous unexpected cherry-pick for `TaskManager` (#2511) 2024-04-07 19:33:54 +08:00
Peng Junzhi 301e99b008 chore: remove required ci with java8 (#2503) 2024-04-03 23:20:40 +08:00
Simon Cheung 7aedba253a chore: add swagger-ui LICENSE relative files (#2495)
* Create LICENSE-swagger-ui.txt
* Update NOTICE
2024-04-03 21:05:47 +08:00
imbajin 2843d4c78e chore: upgrade to 1.3.0 (last major version support Java8)
fix
2024-04-03 21:05:41 +08:00
V_Galaxy 50ae4dee72
chore(pd): refactor and fix pd tests (#2492)
* fix: StoreServiceTest
2024-04-03 19:06:37 +08:00
imbajin 08e0f6486a refact: enhance auth logic
fix
2024-03-22 16:03:53 +08:00
imbajin dcc335fc3e fix: multi graph error in rocksdb with same path/fd 2024-03-22 16:03:50 +08:00
SunnyBoy-WYH 6d04e9b5cb chore(server): clear context after req done (#2470)
Co-authored-by: vaughn.zhang <vaughn.zhang@zoom.us>
Co-authored-by: imbajin <jin@apache.org>
2024-03-19 17:45:24 +08:00
V_Galaxy 67aba571cc
fix(pd-store): intro `Useless` annotation & refactor and mark the failed tests in `hg-pd-test` (#2480)
* add value for Useless annotation

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-19 15:58:56 +08:00
V_Galaxy f34b528885
feat(pd-store): added the OpenTelemetry trace support for pd-store (#2479) 2024-03-16 20:14:09 +08:00
Bond d56ad10105 feat: added the OpenTelemetry trace support (#2477)
TODO: we need enhance our shell experience
---------

Co-authored-by: imbajin <jin@apache.org>
Co-authored-by: VGalaxies <vgalaxies@apache.org>
2024-03-15 20:31:30 +08:00
vaughn f4edfe3692 fix(auth): enhance the URL check (#2422)
Co-authored-by: imbajin <jin@apache.org>
2024-03-09 21:09:33 +08:00
V_Galaxy bdb64767ca fix(server): 'serverStarted' error when execute gremlin example (#2473) 2024-03-05 16:00:44 +08:00
SunnyBoy-WYH c74ec03a27 fix(server): fix the bug which promtheus cant collect hg metric (#2462) 2024-03-05 14:03:00 +08:00
SunnyBoy-WYH 6565a22137 fix(server): add tip for gremlin api NPE with empty query (#2467)
fix #2426
2024-03-02 01:24:23 +08:00
sheli00 a199534b89
fix(pd-store): fix hstore backend api tests failure (#2463) 2024-03-01 17:16:12 +08:00
Hongjun Li 044fd091a6 chore: Add a newline formatting configuration and a comment for warning (#2464)
- Add a newline formatting configuration
- Add a comment for the IDEA unsupported warning about `continuation_indent_size`
2024-02-29 20:39:20 +08:00
Peng Junzhi 6db8b08f14
fix(pd-store): fix hstore backend core tests failure (#2431)
After this commit, the core tests of hstore expect the following tests to fail:

```text
Failures:
  EdgeCoreTest.testScanEdgeInPaging:5214 expected:<18> but was:<0>
  MultiGraphsTest.testCopySchemaWithMultiGraphs:121
  VertexCoreTest.testAddOlapRangeAndOlapSecondaryProperties:2491 expected:<1> but was:<0>
  VertexCoreTest.testAddOlapRangeProperties:2344 expected:<1> but was:<0>
  VertexCoreTest.testAddOlapSecondaryProperties:2254 expected:<1> but was:<0>
  VertexCoreTest.testQueryOlapRangeAndRegularSecondaryProperties:2638 expected:<1> but was:<0>
  VertexCoreTest.testQueryOlapWithUpdates:2725 expected:<1> but was:<0>
  VertexCoreTest.testScanVertexInPaging:7606 expected:<10> but was:<1>
Errors:
  EdgeCoreTest.testQueryOutEdgesOfVertexBySortkeyWithMoreFields:3912 » HgStoreClient
  EdgeCoreTest.testQueryOutEdgesOfVertexBySortkeyWithMoreFieldsInPage:4131 » HgStoreClient
  EdgeCoreTest.testQueryOutEdgesOfVertexBySortkeyWithPrefix:3704 » HgStoreClient
  EdgeCoreTest.testQueryOutEdgesOfVertexBySortkeyWithPrefixInPage:3800 » HgStoreClient
  EdgeCoreTest.testQueryOutEdgesOfVertexBySortkeyWithRange:3607 » HgStoreClient ...
  MultiGraphsTest.testCreateGraphWithSameNameDifferentBackends:289->openGraphWithBackend:381 » Runtime
  RamTableTest.testReloadAndQuery:112 » Huge Failed to load ramtable
  RamTableTest.testReloadAndQueryWithBigVertex:369 » Huge Failed to load ramtabl...
  RamTableTest.testReloadAndQueryWithMultiEdges:239 » Huge Failed to load ramtab...
  RamTableTest.testReloadAndQueryWithProperty:444 » Huge Failed to load ramtable
  RamTableTest.testReloadFromFileAndQuery:159 » Huge Failed to load ramtable
  VertexCoreTest.testQueryByPropertyInPageWithLimitGtPageSize:8337 » IllegalArgument
  VertexCoreTest.testQueryByRangeIndexInPage:8568 » IllegalArgument Undefined in...
  VertexCoreTest.testQueryBySingleRangePropertyInPage:8423 » IllegalArgument Und...
```

---------

Co-authored-by: VGalaxies <vgalaxies@apache.org>
2024-02-27 17:42:59 +08:00
Hongjun Li 198f7004ae chore: refine the hg-style.xml specification (#2457)
- Change the maximum wrap text length for XML to 120 (via `hg-style.xml`)
- Add a blank line between method definitions, inner class definitions, and static code blocks (via `hg-style.xml`)
- Add `<p>` tags to Javadoc blank lines (via `hg-style.xml`)
- `.xml` file leaves a blank line at the end (via `.editorconfig`)
- Class declaration requires a blank line before and after each line (via `hg-style.xml`)
- Javadoc line comment not in line (via `hg-style.xml`)
- `.properties` file leaves empty lines (via `hg-style.xml`)
- `.yaml` file braces `{}`, brackets `[]` remove spaces (via `hg-style.xml`)
2024-02-27 12:55:17 +08:00
Z-HUANT 3b9507001e feat(api): optimize adjacent-edges query (#2408)
Relevant issue: #2255

Gremlin Query: For adjacency edge queries, if a vertex does not belong to the adjacent vertices of this edge, filter out that vertex.

---------

Co-authored-by: imbajin <jin@apache.org>
2024-02-26 18:25:41 +08:00
Jermy Li 74cf3109ec chore: unify to call SchemaLabel.getLabelId() (#2458)
Change-Id: I31bcc0d1ee99f3c443f8f4f0d458e06ca89977ef
2024-02-25 15:08:07 +08:00
M 02c9b4eac0 fix(server): remove extra blank lines (#2459)
* remove extra blank lines in api

* remove extra blank lines in core

* remove extra blank lines in scylladb
2024-02-25 15:07:28 +08:00
V_Galaxy 9fc3788d9c chore: improve license header checker confs and pre-check header when validating (#2445)
* improve license header checker confs

* typo

* verify -> validate

* update rat exclude files

* empty commit

* update rat exclude files

* update rat exclude files
2024-02-23 16:58:01 +08:00
V_Galaxy 168cf69ac7 fix: protobuf file header (#2448) 2024-02-23 16:46:52 +08:00
M b7f8b56730 fix(server): clean up the code (#2456)
- replace `size() == 0` with `isEmpty()`
- remove unnecessary unboxing
- remove unnecessary boxing
- replace for loop with enhanced for loop
- replace lambda with method reference
2024-02-22 17:28:43 +08:00
M 5551ec17f2 fix: format and clean code in modules (#2439)
Format & clean code in submodels:

1. API
2. Scylladb
3. Postgresql
4. Rocksdb
5. Palo
6. Mysql
7. Hbase
8. Cassandra
9. Test

---------

Co-authored-by: imbajin <jin@apache.org>
2024-02-20 21:55:19 +08:00
M 960b7330fd fix: format and clean code in core module (#2440) 2024-02-20 21:54:58 +08:00
M 721111d11a fix: format and clean code in dist and example modules (#2441) 2024-02-20 18:55:58 +08:00
V_Galaxy d61aaae44f
fix(pd-store): correct license in hugegraph-pd (#2446)
* fix: correct license in hugegraph-pd

* minor fix

* add header for protobuf files
2024-02-20 18:28:35 +08:00
V_Galaxy e4bd6446f0
fix(pd-store): correct license in hugegraph-store (#2447)
* fix: correct license in hugegraph-store

* add header for protobuf files

* remove mvnw
2024-02-20 18:25:48 +08:00
M 92f2ecb5e7 fix(server): unify the license headers (#2438)
subtask of #2435

---------

Co-authored-by: imbajin <jin@apache.org>
Co-authored-by: VGalaxies <vgalaxies@apache.org>
2024-02-07 13:41:00 +08:00
VGalaxies 7b14857828 fix: construct PdMetaDriver and pd-store log confs 2024-02-04 11:37:39 +08:00
V_Galaxy bc9c1e7ac9
chore(pd-store): minor improvement (#2443) 2024-02-04 11:17:06 +08:00
VGalaxies d4a20a120a chore: sync adapted code to internal 4.0 version
only the code within hugegraph-core module
2024-02-03 22:04:21 +08:00
Peng Junzhi b06779aed6 fix(server): make CacheManager constructor private to satisfy the singleton pattern (#2432) 2024-02-03 22:03:14 +08:00
SunnyBoy-WYH ec3779c709 chore(server): update swagger info for default server profile (#2423) 2024-01-29 23:25:22 +08:00
Dandelion 70150ad448 fix(chore): remove zgc in dockerfile for ARM env (#2421)
* remove zgc

* Apply suggestions from code review

Co-authored-by: imbajin <jin@apache.org>

* add comment for hugegraph-server.sh

* fix enable-auth.sh

* init store in entrypoint

* use flag file to skip re-init

* delete tar.gz

* simply dockerfile

* mvn optimize

* simply dockerfile

* add init log in docker

---------

Co-authored-by: imbajin <jin@apache.org>
2024-01-29 23:24:25 +08:00
V_Galaxy 46cb612eb5
fix(pd-store): partially fix hstore backend core tests failure (#2424)
Some tests that are currently failing (not addressed in this PR):

1. Tests related to **olap** in `VertexCoreTest`: Not adapted yet.
2. Tests related to **paging** in `VertexCoreTest` and `EdgeCoreTest`: Inconsistency in the definition of `position` between RocksDB scan iterator and Hstore.
3. Some `RamTableTest` test cases: The ramtable feature is not supported by the hstore backend.
2024-01-29 22:49:48 +08:00
Peng Junzhi 499f8e74b5
fix(server): starting error of server (#2419)
1. When starting hugegrpah-server in idea, we will encounter an error: `java.lang.ClassNotFoundException: io.grpc.stub.AbstractBlockingStub`. We should remove the `<scope>` line in pom to avoid that.

2. Add dependency support for arm-architecture machine
2024-01-18 12:15:26 +08:00
Z-HUANT 12ff9154ab fix(server): reinitialize the progress to set up graph auth friendly (#2411) 2024-01-17 23:56:27 +08:00
Dandelion 94a79326ea feat: support docker use the auth when starting (#2403)
- allow user to set env for docker to set auth mode
- download keystore when package
- fix a curl error (also use curl first in `function` download)

---------

Co-authored-by: imbajin <jin@apache.org>
2024-01-16 02:00:49 +08:00
V_Galaxy e11dc8e5f3
chore(pd-store): cherry-pick commits from master for synchronization & some modifications for passing partial hstore tests (#2414) 2024-01-16 01:21:27 +08:00
VGalaxies dd44f701f5 fix: TaskScheduler.delete test caller sets the force parameter to false 2023-11-10 19:45:41 +08:00
V_Galaxy 8b3624aa03 refact: adjust project structure for merge PD & Store[Breaking Change] (#2338)
Subtask of #2265.

Adjust the project structure of this repository to include three sub-modules: hugegraph-server, hugegraph-pd, hugegraph-store at the root level.

Roll back to the moment when https://github.com/apache/incubator-hugegraph/pull/2266 was merged on `pd-store` and incorporate the latest changes in `master`.

For more detailed information, please refer to https://github.com/apache/incubator-hugegraph/pull/2266#issue-1834369489.

---------

Co-authored-by: M <87920097+msgui@users.noreply.github.com>
2023-11-10 19:45:41 +08:00
SunnyBoy-WYH 896d943538 feat(api): support recording slow query log (#2327)
* chore(api): code style for cr

---------

Co-authored-by: imbajin <jin@apache.org>
2023-11-10 19:45:41 +08:00
SunnyBoy-WYH 73bf6607fd chore(api): add swagger desc for Arthas & Metric & Cypher & White API (#2337)
add swagger belong for arthas API

---------

Co-authored-by: imbajin <jin@apache.org>
2023-11-10 19:45:41 +08:00
lzyxx ee0299e6d3 feat(cassandra): adapt cassandra from 3.11.12 to 4.0.10 (#2300) 2023-11-10 19:45:41 +08:00
VGalaxies 785d387294 chore: move RamIntObjectMapTest.java 2023-11-10 19:45:41 +08:00
conghuhu e705740e1e fix(core): handle schema Cache expandCapacity concurrent problem (#2332) 2023-11-10 19:45:41 +08:00
Wu Chencan 94edfcd720 feat(core): support batch+parallel edges traverse (#2312)
- Enhance Consumers.java, supporting ExceptionHandle and `Future` to handle InterruptedException when awaiting
- Add Nested Iterator Edge and support batch execution
- Support batch execution & thread parallel in KoutTraverser and Kneighbor
2023-11-10 19:45:41 +08:00
Dandelion 263bb2feb4 fix: always wait for storage if rocksdb is selected (#2333) 2023-11-10 19:45:41 +08:00
Dandelion 87509b8a99 feat: support Cassandra with docker-compose in server (#2307)
1. change the dockerfile, adding the shell to wait for storage backend and use a docker-entrypoint.sh to manage the starting process.
2. delete a deprecated class in  gremlin-console.sh (reference: [doc of ScriptExecutor](https://tinkerpop.apache.org/javadocs/3.2.3/full/org/apache/tinkerpop/gremlin/groovy/jsr223/ScriptExecutor.html))
3. add a healthy check in docker-compose
4. add an example folder where we can put all the template docker-compose.yml here
5. add `*swagger-ui*` in gitignore, which appears after you compile the source code locally.

---------

Co-authored-by: imbajin <jin@apache.org>
2023-11-10 19:45:41 +08:00
Dandelion 785afe05ba doc: README.md tiny improve (#2331) 2023-11-10 19:45:41 +08:00
SunnyBoy-WYH 8115325d9a feat(api): support embedded arthas agent in hugegraph-server (#2278) 2023-11-10 19:45:41 +08:00
Jermy Li 5a064de05a README.md tiny improve (#2320) 2023-11-10 19:45:41 +08:00
SunnyBoy-WYH 176637c47a feat(api): support metric API Prometheus format & add statistic metric api (#2286) 2023-11-10 19:45:41 +08:00
SunnyBoy-WYH 4bb48814bb feat: support White IP List (#2299)
tips:
- this feat works when auth mode was set.
- this feat works when white ip status was enabled.

because now PD is unavailable,just use java list; when pd ready , we can checkout pd.
2023-11-10 19:45:41 +08:00
Dandelion b116fc8a21 doc: update README about start server with example graph (#2315) 2023-11-10 19:45:41 +08:00
M 10c570d368 fix: base-ref/head-ref missed in dependency-review on master (#2308) 2023-11-10 19:45:41 +08:00
Wu Chencan 9c2d3ad6d3 feat(api-core): support label & property filtering for both edge and vertex & support kout dfs mode (#2295)
- Support label & property filtering for both edge and vertex and the filtering is implemented in Kout Post and Kneighbor - Post Apis, reducing unnecessary graph searches through pruning
- Support Kout dfs mode in Kout Post Api

Originally only edge label filtering was supported, now label and property filtering for edge and vertex is supported.
- add classes VEStepEntity and VEStep to support serialization in request
- add class Steps to support filtering of edge and vertex in runtime(core)
- add new method edgesOfVertex(Id source, Steps steps) to support label and property filtering for both edge and vertex in HugeTraverser.java

---------

Co-authored-by: imbajin <jin@apache.org>
2023-11-10 19:45:41 +08:00
Dandelion 4ad54181b4 fix(dist): avoid var PRELOAD cover environmnet vars (#2302)
Update hugegraph-dist/src/assembly/static/bin/start-hugegraph.sh

---------

Co-authored-by: imbajin <jin@apache.org>
2023-11-10 19:45:41 +08:00
Chong Shen 51df7ec871 fix(core): close flat mapper iterator after usage (#2281)
close [Bug] FlatMapperIterator should be closed after usage #2280
2023-11-10 19:45:41 +08:00
Dandelion 1bac480710 feat(dist): support pre-load test graph data in docker container (#2241)
- Provide the related conf and groovy for user to pre load some data.
- Change the start-hugegraph.sh to get the environment variables to decide to pre-load or not.

---------

Co-authored-by: imbajin <jin@apache.org>
2023-11-10 19:45:41 +08:00
V_Galaxy d5fa5c8e33 chore(dist): replace wget to curl to download swagger-ui (#2277)
Main Changes:
1. replace `wget` by `curl` when downloading `swagger-ui`
2. silence the output of `curl` and `tar` commands
3. reuse the existing `v4.15.5.tar.gz` before downloading
4. avoid downloading `swagger-ui` in non-Linux platforms to prevent build failures (there might be a cross-platform build approach available 🤔)
5. wrapp the script content within `<![CDATA[ ... ]]>` blocks ensures that the script retains its original format when generating the `dist.sh` script (also suppresses automatic indentation)
6. remove intermediate files at the end of the script

**An alternative approach**, during the generation of the `dist.sh` script, only the `${final.name}` property from the build process is utilized. It might be possible to separately store a `dist.sh` script within hugegraph-dist, then use `sed` during the build process to replace the value of `${final.name}`, **thereby avoiding the need to embed script content within the pom file**.

---------

Co-authored-by: imbajin <jin@apache.org>
2023-11-10 19:45:41 +08:00
lzyxx 07b7243efe fix checkstyle: Update StandardStateMachineCallback.java (#2290)
During the compilation of your code, an informational message was displayed indicating an issue with the file /home/lzy/hugegraph/hugegraph-core/src/main/java/org/apache/hugegraph/masterelection/StandardStateMachineCallback.java at line 36. The specific problem was that the length of this line exceeded 100 characters, with a total of 101 characters.

To address this issue, I have made modifications to this class. I have split the originally long line into multiple lines to ensure that each line's length adheres to the coding standards' specified limits. This action not only aligns with the requirements of the code style, but also improves the readability and maintainability of the code.
2023-11-10 19:45:41 +08:00
DanGuge c7054e0704 feat(api&core): in oltp apis, add statistics info and support full info about vertices and edges (#2262)
* chore: improve gitignore file

* feat: add ApiMeasure to collect runtime data

ApiMeasure will count the number of vertices and edges traversed at runtime, and the time the api takes to execute

* feat: Add ApiMeasure to JsonSerializer and Modify the Serializer interface

* JsonSerializer: return measure information in api response

* Serializer: fit the feature that returns complete information about vertices and edges

* refactor: format code based on hugegraph-style.xml

* feat: Add statistics information in all oltp restful apis response and Support full information about vertices and edges

Statistics information:

* add vertexIterCounter and edgeIterCounter in HugeTraverser.java to track traversed vertices and edges at run time

* modify all oltp restful apis to add statistics information in response

Full information about vertices and edges:

* add 'with_vertex' and 'with_edge' parameter option in apis

* modify oltp apis to support vertex and edge information in api response

* add EdgeRecord in HugeTraverser.java to record edges at run time and generate the edge information returned in api response

* modify Path and PathSet in HugeTraverser.java to support full edge information storage

* modify all traversers to support track of edge information at run time

* fix: numeric cast

* fix: Jaccard Similarity api test

* fix: adjust the code style and naming convention

* Empty commit

* Empty commit

* fix:
1. change System.currentTimeMillis() to System.nanoTime();
2. modify addCount()

* fix: rollback change in .gitignore

* fix: rollback ServerOptions.java code style

* fix: rollback API.java code style and add exception in else branch

* fix: fix code style

* fix: name style & code style
* rename edgeRecord to edgeResults
* fix Request class code style in SameNeighborsAPI.java
2023-11-10 19:45:41 +08:00
V_Galaxy c13b4e2b3a
fix: check leader role before setPdConfig (#2341) 2023-11-03 17:09:45 +08:00
Simon Cheung 823b1902eb
fix(pd): log4j version conflict & follower NPE (#2339)
* fix: log4j version conflict

* Update RaftRpcClient.java
2023-10-30 16:32:59 +08:00
V_Galaxy fe7697e9a7
chore: enhance a string of problems to pass CI (#2321)
## Main Changes

- fix license header (from https://github.com/apache/incubator-hugegraph/pull/2303)
	- remove maven wrapper in pd and store
- remove java 8 CI in `ci.yml` (not support java 8 in HugeGraph 1.5.0)
- add `pd-store.yml` for pd, store, hstore UT
- collect the test files from pd into the `hg-pd-test` module, the same for store
	- except for `hg-pd-service`, `hg-store-node`, `hg-store-rocksdb`

## TODO

- [ ] dependency-check (update the `known-dependencies.txt` file in `hugegraph-server/hugegraph-dist/scripts/dependency/`)
	- [ ] With the change in project structure, some scripts in `hugegraph-server/hugegraph-dist` may need to be relocated to the appropriate locations. 🤔
- [ ] `hugegraph-server/hugegraph-dist/src/assembly/jenkins/`
2023-10-16 16:01:10 +08:00
V_Galaxy f34743b5fb
feat: preliminary support for distributed task scheduler (#2319)
* chore: migrate DistributedTaskScheduler

* fix: basic setup for DistributedTaskScheduler

* chore: set default scheduler type to local

* chore: code style

* chore: code style

* fix: impl checkRequirement for DistributedTaskScheduler

* chore: add comment for task server id

* fix: task result vertex type

* chore: review

* fix: StandardClusterRoleStore queryVertex

* chore: add comment

duplicated commit in hstore for clusterRole
2023-10-16 11:22:50 +08:00
V_Galaxy b670c01002
feat: preliminary support for hstore backend (#2301)
* chore: move hugegraph-hstore

* chore: simple adapt

* chore: register hstore

* build: intro grpc dep

* chore: migrate org.apache.hugegraph.meta

* chore: migrate org.apache.hugegraph.space

* chore: adapt meta schema

* chore: migrate SchemaTransactionV2

* chore: adapt SchemaTransactionV2

* refact: intro ISchemaTransaction for adaptation

* fix: init meta server

* fix: avoid typecast from UnmodifiableRandomAccessList

class java.util.Collections cannot be cast to class java.util.ArrayList

* refact: query task and server info by graph tx

* fix: register task and server table for rocksdb BE

* chore: add example for hstore BE scanning tables

* fix: register task and server table for other BEs

* build: remove system dependency to hugegraph-core in hg-store-core

* chore: remove author comments

* chore: remove AbstractDistributedLock class and lock method

* chore: replace HashMap with ImmutableMap

* chore: code review

* chore: load driver version in graph store for hstore

* fix: index serialization and deserialization logic

* fix: convertTaskOrServerToVertex for other BEs

* chore: init meta server in ctor of StandardHugeGraph

* chore: add comments for lock module

---------

Co-authored-by: heiyan <heiyan2020@gmail.com>
2023-09-18 17:17:37 +08:00
M 0903e900e4
feat: support visit of stage repository (#2305)
* add: stage repository

* feat: enable user access to stage repo

* feat: support jdk8 compilation, exclude jdk11-only modules

* fix:  jdk version

* fix:apply suggestions

Co-authored-by: imbajin <jin@apache.org>

* Update pom.xml

---------

Co-authored-by: imbajin <jin@apache.org>
2023-09-06 22:22:27 +08:00
V_Galaxy c70a2944b0
fix: assemble executable jar for pd and store (#2289)
Due to the change of hugegraph-pd's parent from spring-boot-starter-parent to hugegraph, and hugegraph's parent being apache, along with modifications to the artifactId of certain sub-modules (for example, hugegraph-pd to hg-pd-service), the following steps need to be taken:

Modify the descriptor of the maven-assembly-plugin, which involves adjusting the collected jar package names accordingly.
- Update the corresponding jar package names in the startup script.
- Since hugegraph-pd is started using Spring Boot, the spring-boot-maven-plugin needs to be reconfigured.

---------

Co-authored-by: imbajin <jin@apache.org>
2023-08-29 18:43:36 +08:00
M d6a2548427
fix(server): github ci for submodule "server" (#2288) 2023-08-21 19:15:00 +08:00
Yuexuan Zhang b19aa30454
refact: adjust pom structure for directly compiling from root (#2275)
* build: remove unused deps in hugegraph-pd

* chore: add compiled hugegraph-core.jar as dependency

* chore: remove unused dependency.

* fix: change hugegraph-core scope type in hg-store-node

* build: unify revision to 1.5.0.1

* chore: clean swagger files

---------

Co-authored-by: V_Galaxy <1904821183@qq.com>
Co-authored-by: imbajin <jin@apache.org>
2023-08-11 18:36:53 +08:00
V_Galaxy dfd3c6700e
refact: merge store into hugegraph (#2272)
also fix maven-clean-plugin dist dir clean
2023-08-08 17:41:36 +08:00
imbajin 7e9b8afbc6
refact(store): merge branch 'pr/2272' into pd-store 2023-08-08 17:20:34 +08:00
imbajin 4f34d01eb2
refact(pd): merge branch 'pr/2270' into pd-store 2023-08-08 17:11:46 +08:00
imbajin eb76bc62a0
chore: allow merge for a string of commits/PR 2023-08-08 10:41:58 +08:00
imbajin c72bda6237
Merge branch 'pd-store' into merge-store 2023-08-07 23:26:28 +08:00
V_Galaxy a891780c36
refact: merge pd into hugegraph (#2270)
* init(pd): history merged until 2023.5.6

* refact: unify LF line separator

Change-Id: I3f38685af534468a51b79b7f45d24fdb30a74f34

* refact: add header & format in test & service & grpc & dist modules

Change-Id: Id2fa1374d7fa85635906a7e75655e7dad56f1807

* refact: add header/format in core & common & clitools & client modules

Change-Id: Id8ec0a0d754f646c59f7251158dd89de669b2016

* refact: format root/parent modules & pd-service package name & rearrange code

Change-Id: Id8ec0a0d754f646c59f7251158dd89de669b2016

* chore: search & replace to "org.apache" manually

- also replace "a/b/c" regrex
- replace pom version to ${revision}

Change-Id: Ia3a9ce891fcff58cef4a5a1f95adb44ef8ee8049

* chore: fix some code problems

- lack this with instance filed
- lack {} in 'if/while/for'

Change-Id: I37d21f1ab94529a8122f4598636fbe02b2264904

* refact: update code to Jun2 2023

GraphPlatform-2020 fix cache bug when direct put

Change-Id: I52428d8d5d4bb977ee7796c0fe1e4ced07aff12d

GraphPlatform-2020 fix cache bug when direct put

Change-Id: Id57427a94732e12f2d96429a2b677d5c5f71e8ff

GraphPlatform-1771 update code

Change-Id: Ic3b95ca22900453adbc3571734e1da2c97e28355

* build: add flatten plugin and fix dependency version

1. Added the
[flatten](https://github.com/apache/incubator-hugegraph/pull/2004)
plugin.
2. Unified the global version to 1.5.0.
3. Modified the version of `jraft-core`.

---------

Co-authored-by: imbajin <jin@apache.org>

* build: fix build for community edition compatibility (WIP)

Please refer to
https://hugegraph.feishu.cn/wiki/Y6d2wys9KiWf24kpzNKct0Yknnr for
detailed fix documentation.

---------

Co-authored-by: imbajin <jin@apache.org>

* refact: adapt for jraft v1.3.13

* refact: GP-2141 handle low speed import - 6.9

from inner commit 0a95b1e587e65ea6a0a06774d53c80868f36af5b

---------

Co-authored-by: V_Galaxy <1904821183@qq.com>

* refact: GP-2141 handle low speed import - Compile OK

from inner commit 0a95b1e587e65ea6a0a06774d53c80868f36af5b

fix store compile problem
https://hugegraph.feishu.cn/wiki/Y6d2wys9KiWf24kpzNKct0Yknnr

---------
Co-authored-by: V_Galaxy <1904821183@qq.com>

* chore: enhance pom & start file

---------

Co-authored-by: V_Galaxy <1904821183@qq.com>

* refact: prepare before merge

* chore: fix pom for submodules in hugegraph-pd

* chore: add apache-rat-plugin for hugegraph-pd

and add some license headers

* chore: clean some comment/code

---------

Co-authored-by: imbajin <jin@apache.org>
Co-authored-by: What is broken can be reforged <34939118+GYXkeep@users.noreply.github.com>
2023-08-07 23:15:28 +08:00
V_Galaxy 84273f2093
chore: clean some comment/code 2023-08-07 18:07:00 +08:00
V_Galaxy 497d62eefe chore: clean some comment/code 2023-08-07 17:07:26 +08:00
V_Galaxy 1234c110e7 chore: add apache-rat-plugin for hugegraph-pd
and add some license headers
2023-08-07 16:09:32 +08:00
V_Galaxy cbdd65a06f chore: fix pom for submodules in hugegraph-pd 2023-08-07 16:08:57 +08:00
V_Galaxy e78eb82b6a chore: add apache-rat-plugin for hugegraph-store
and add some license headers
2023-08-07 15:31:25 +08:00
V_Galaxy 02c36d848e chore: fix pom for submodules in hugegraph-store 2023-08-07 15:22:14 +08:00
V_Galaxy 51ca58746d Merge branch 'store' into merge-store 2023-08-07 15:06:00 +08:00
V_Galaxy 8923b5d6e0 refact: prepare before merge 2023-08-07 15:02:59 +08:00
V_Galaxy 84d8beb7e0 fix: mismatch table name when parsing BatchEntry
fix https://github.com/hugegraph/store/pull/4

When the server uses hstore as the backend, taking writing a vertex as
an example. The server will construct an `HgStoreSession` using
graphspace and graph, like `DEFAULT/hugegraph/g`.

Then it calls the `put` method of `HgStoreSession`, passing the
corresponding table (here it is `g+v`), key, and value.

```java
@Override
public void put(String table, byte[] ownerKey, byte[] key, byte[] value) {
    prepare();
    this.graph.put(table, HgOwnerKey.of(ownerKey, key), value);
}
```

The store client will encapsulate this write operation as an RPC request
in the following format:

```protobuf
message BatchEntry {
  OpType op_type = 1;
  int32 table = 2; // NOTE HERE
  Key start_key = 3;
  Key end_key = 4;
  bytes value = 5;
}
```

https://github.com/hugegraph/store/pull/4 actually fixes the logic that
maps `g+v` to the corresponding code.

However, after receiving `BatchEntry`, the store still needs to parse
the corresponding table name based on the code.

```java
default void doBatch(String graph, int partId, List<BatchEntry> entryList) {
    BusinessHandler.TxBuilder builder = txBuilder(graph, partId);
    try {
        for (BatchEntry b : entryList) {
            Key start = b.getStartKey();
            String table = tables[b.getTable()]; // NOTE HERE
```

This step was ignored in https://github.com/hugegraph/store/pull/4,
resulting in the actual table being written into the store as `vertex`
(the old table name).
2023-08-07 14:55:03 +08:00
imbajin 61a8bedd70 refact(dist): enhance start-hugegraph-store.sh 2023-08-07 14:55:03 +08:00
imbajin 28bb45d60c fix(client): mismatch store table name when put data
`prepareBatchEntry()` throws NPE, refer
https://hugegraph.feishu.cn/wiki/O07Bw2jnPiJGZbkE9utcw65Xntc

thanks @VGalaxies to fix this problem

Co-authored-by: V_Galaxy <1904821183@qq.com>
2023-08-07 14:55:00 +08:00
imbajin 7b56c40650 fix: update pom config & start script 2023-08-07 14:55:00 +08:00
V_Galaxy cffa495446 build: fix build for community edition compatibility (WIP)
Please refer to
https://hugegraph.feishu.cn/wiki/Y6d2wys9KiWf24kpzNKct0Yknnr for
detailed fix documentation.

---------

Co-authored-by: imbajin <jin@apache.org>
2023-08-07 14:54:55 +08:00
What is broken can be reforged 0cad1fdb45 refact: adopt jraft to open source version
address the
[jraft-doc](https://hugegraph.feishu.cn/wiki/JGy0wPlCsih9XMkjrNMcNfhxnBb)
2023-08-07 14:54:51 +08:00
V_Galaxy 8c03d72744 chore: add flatten plugin and fix dependency version
* build: add flatten plugin and fix dependency version

* Update pom.xml

---------

Co-authored-by: imbajin <jin@apache.org>
2023-08-07 14:54:17 +08:00
V_Galaxy 3910af8f5b Merge branch 'pd' into merge-pd 2023-08-05 23:42:56 +08:00
V_Galaxy 49a1acf3b2 fix: fix parent of submodules in hugegraph-server
modify pom description as well
2023-08-05 23:24:33 +08:00
V_Galaxy fd161f2f15 refact: init root pom for all modules 2023-08-05 23:24:33 +08:00
V_Galaxy 9068cef7e8 refact: create dirs for hugegraph-pd and hugegraph-store 2023-08-05 23:24:33 +08:00
V_Galaxy 3559aced8c refact: move dirs into hugegraph-server 2023-08-05 23:24:33 +08:00
V_Galaxy 7c325c5efd refact: prepare before merge 2023-08-03 19:56:42 +08:00
imbajin a37177207c chore: enhance pom & start file
---------

Co-authored-by: V_Galaxy <1904821183@qq.com>
2023-08-03 19:56:36 +08:00
imbajin ec0b0e006b refact: GP-2141 handle low speed import - Compile OK
from inner commit 0a95b1e587e65ea6a0a06774d53c80868f36af5b

fix store compile problem
https://hugegraph.feishu.cn/wiki/Y6d2wys9KiWf24kpzNKct0Yknnr

---------
Co-authored-by: V_Galaxy <1904821183@qq.com>
2023-08-03 19:56:28 +08:00
imbajin 3ea878c3a8 refact: GP-2141 handle low speed import - 6.9
from inner commit 0a95b1e587e65ea6a0a06774d53c80868f36af5b

---------

Co-authored-by: zhangyingjie02 <zhangyingjie02@baidu.com>
Co-authored-by: V_Galaxy <1904821183@qq.com>
2023-08-03 19:56:21 +08:00
What is broken can be reforged 6092dc0fc1 refact: adapt for jraft v1.3.13 2023-08-03 19:56:15 +08:00
V_Galaxy ed00d4eb29 build: fix build for community edition compatibility (WIP)
Please refer to
https://hugegraph.feishu.cn/wiki/Y6d2wys9KiWf24kpzNKct0Yknnr for
detailed fix documentation.

---------

Co-authored-by: imbajin <jin@apache.org>
2023-08-03 19:56:03 +08:00
V_Galaxy 8ddf8291af build: add flatten plugin and fix dependency version
1. Added the
[flatten](https://github.com/apache/incubator-hugegraph/pull/2004)
plugin.
2. Unified the global version to 1.5.0.
3. Modified the version of `jraft-core`.

---------

Co-authored-by: imbajin <jin@apache.org>
2023-08-03 19:55:52 +08:00
imbajin 6a813d2c06 update code to Jun2 2023
Change-Id: I9a21c17a07c96c42eafc5ffeedfc70b2a77563d2

GraphPlatform-2120 检查channel

Change-Id: I9135b19b2a3bd66f31dcdbcf9169b1884ae934c2

GraphPlatform-2120 处理加锁

Change-Id: I64441d02221e96aa21027faafe90c8b7e04b15d1

GraphPlatform-1749 修复可能导致的死锁问题

Change-Id: Ie5ab9fd62746bfcd77edcaf8881671abee678d31

GraphPlatform-1749 修正pd raft

Change-Id: I31064fa2ce31b677cf044a85645e7244cbbbff26

GraphPlatform-2120 控制pd线程池大小

Change-Id: Ie505f5525bcff79f84c0cd674b92c6abf3a2c03f

GraphPlatform-1749 优化partition cache

Change-Id: I5cafd9078ccc91b7f0b4d9157a3913049c38d8cb

GraphPlatform-1657 rest接口转发到leader

Change-Id: Ib5db654a35733ddead7972d5e4b08b981a89fb31

GraphPlatform-2020 fix cache bug when direct put

Change-Id: I52428d8d5d4bb977ee7796c0fe1e4ced07aff12d

GraphPlatform-2020 fix cache bug when direct put

Change-Id: Id57427a94732e12f2d96429a2b677d5c5f71e8ff

GraphPlatform-1771 update code

Change-Id: Ic3b95ca22900453adbc3571734e1da2c97e28355
2023-06-20 19:51:09 +08:00
imbajin 1252f5097c refact: update the code to 2023.6.19
Change-Id: Ib861b47cab873322d9427d6a1f654f5cae92ddb6

GraphPlatform-1749 修复显示的版本号

Change-Id: I09b072b802d60bf1820f5a6dd18e2570eee4835b

GraphPlatform-1657 修改grpc消息大小参数设置

Change-Id: Ifb51b841c5be7690407cc90b34a72d4b2bdea360

GraphPlatform-1595 调整ci及减少锁粒度

Change-Id: I496c814cac580a51125e382b538eaf57445121dc

FIN GraphPlatform-2141 增加按表查询接口

Change-Id: Iee2d3503d2a7fd4815ecf7dec828afa28a24e46a

GraphPlatform-2147 迭代完成及时释放rocksdb迭代器,部分并发控制改进

Change-Id: I69f3985fd7a29b718f10eb6718488d63e6a1a077

GraphPlatform-2161 PD leader变更,客户端重连

Change-Id: Ibfeb837e94ecc9dfc19e6fdc85b421df725a6cb7

FIN GraphPlatform-2147 处理并发

Change-Id: I7a41969a8689eb216d2a7537a7a2f94846427c0e

FIN GraphPlatform-2140 修改并发控制

Change-Id: I89cbee229c111716b092a6df903aa692be03b9bb

FIN GraphPlatform-2140 并发控制改进

Change-Id: Ie9c7755b330d2df653c7e93b5592a514bb7aa099

GraphPlatform-2119 修改线程默认配置

Change-Id: Ia6107168b381562a767e34443328391e5da1e1a0
2023-06-20 19:07:36 +08:00
imbajin 9ccbeb7de7 fix: delete duplicate file in common commit & fix org name
Change-Id: I8279b2548a7f16599e61f42281f5aea7ceae51a8
2023-05-09 23:56:32 +08:00
chengxin05 33749fddaf GraphPlatform-2060 更改store依赖common更改为开源的org.apache.hugegraph.common包
Change-Id: Ie3997bad153358f251b242cae8de13ff8022976e
2023-05-09 23:45:28 +08:00
imbajin bd3d313a1b chore: fix some code problems
- lack this with instance filed
- lack {} in 'if/while/for'

Change-Id: I37d21f1ab94529a8122f4598636fbe02b2264904
2023-05-06 20:04:30 +08:00
imbajin 4166b6f8ff chore: search & replace to "org.apache" manually
- also replace "a/b/c" regrex
- replace pom version to ${revision}

Change-Id: Ia3a9ce891fcff58cef4a5a1f95adb44ef8ee8049
2023-05-06 19:57:29 +08:00
imbajin fd13bb436b refact: format root/parent modules & pd-service package name & rearrange code
Change-Id: Id8ec0a0d754f646c59f7251158dd89de669b2016
2023-05-06 19:50:02 +08:00
imbajin 61ecb66e8f refact: add header/format in core & common & clitools & client modules
Change-Id: Id8ec0a0d754f646c59f7251158dd89de669b2016
2023-05-06 19:44:31 +08:00
imbajin dd49f3f54b refact: add header & format in test & service & grpc & dist modules
Change-Id: Id2fa1374d7fa85635906a7e75655e7dad56f1807
2023-05-06 19:41:10 +08:00
imbajin 08f560f388 refact: unify LF line separator
Change-Id: I3f38685af534468a51b79b7f45d24fdb30a74f34
2023-05-06 19:21:16 +08:00
V_Galaxy d97ba4cb51 history merged until 2023.5.6 2023-05-06 18:24:23 +08:00
imbajin e79d5f63a1 fix(sec): upgrade version for super high CVE on springboot & mark license TODO
Change-Id: Id30547444de6bea148efd88d51e88cc866f44047
2023-05-05 14:35:22 +08:00
imbajin 7175358fa6 chore: add .gitattributes file to unify line separator & clean pom
Change-Id: I303c98351c2dafda4fbb279dd8dc1056164b8f70
2023-05-05 13:48:09 +08:00
imbajin 34d7516618 refact: remove inner info
Change-Id: I72b468d3aa247ba969eb28a155576a773f2ea084
2023-04-26 13:18:42 +08:00
imbajin a90fc30cde refact: rearrange code position
Change-Id: I3d559de5acadb1815f10622120f1b4f27a740ff1
2023-04-26 13:00:11 +08:00
imbajin c4f4a3c877 fix: lack override annotations & explicit type & CONSTANTs
Change-Id: Iabac67f3beb848567e940647ffc578af56936c7d
2023-04-26 12:50:38 +08:00
imbajin 082d3399de fix: lack brace for if statement & rearrange code
Change-Id: I2ca0b6d5d30740b41a3bcb9db11e50feb32335cd
2023-04-26 12:14:55 +08:00
imbajin 6bce8990af refact: fix all CRLF files to LF together (Breaking Change)
GraphPlatform-1906 梳理入库与查询流程

Change-Id: I5e7f57893c904e50be320e263a01283b147545a8
2023-04-26 11:12:02 +08:00
imbajin 740563bc61 doc: add LICENSE & NOTICE file in root & dist module
Change-Id: I278a4b7bc669270286548959a0952389a49e3371
2023-04-26 11:10:22 +08:00
imbajin 4ec8c3a0f6 refact: add license header for root files & protobuf files
Change-Id: Id277df46c88cc75029b0b3510274df14f732faba
2023-04-26 11:10:09 +08:00
imbajin 0a60f24f9c refact: full search & replace package name
Change-Id: I8f9780523f1e6d022e66f29d36c3deb12be23845
2023-04-26 11:09:41 +08:00
imbajin 7b5c5f87b7 refact: rename pkg for client & common & core & pom.xml
Change-Id: Ieef748324b11b9d125dca915262f80a5954f5f98
2023-04-26 11:09:24 +08:00
imbajin bb081bcfc2 refact: rename package for cli & dist & node & rocksdb & test modules
TODO: some files need update manully
Change-Id: If83118e2dd123a7a88f9c5f7b33fbc7cf1b55de7
2023-04-26 11:08:51 +08:00
imbajin d62386b5a3 refact: handle rpc package alone for other modules
Change-Id: Icd740de03f9572d8d515c3d2237c2c6cb8318af3
2023-04-26 11:08:15 +08:00
V_Galaxy 03b6d3761b history merged until 2023.4.23 2023-04-23 10:41:18 +08:00
951 changed files with 4636 additions and 38544 deletions

View File

@ -28,21 +28,20 @@ github:
del_branch_on_merge: true
#labels:
enabled_merge_buttons:
merge: false
merge: true
rebase: true
squash: true
protected_branches:
master:
required_status_checks:
# strict means "Require PR to be up-to-date before merging". (enable when branch unstable)
# strict means "Require branches to be up-to-date before merging".
strict: false
# contexts are the names of checks that must pass (now only enable the basic check)
contexts:
- Analyze (java)
- CodeQL
- check-license
- build-server (memory, 11)
- build-commons (11)
- build (memory, 11)
required_pull_request_reviews:
dismiss_stale_reviews: true
require_code_owner_reviews: false

View File

@ -21,10 +21,6 @@ root = true
charset = utf-8
end_of_line = lf
insert_final_newline = true
max_line_length = 100
ij_wrap_on_typing = true
ij_visual_guides = 100
[*.{java,xml,py}]
indent_style = space
@ -36,64 +32,3 @@ continuation_indent_size = 8
[*.md]
max_line_length = off
[*.java]
ij_java_doc_add_p_tag_on_empty_lines = true
ij_java_doc_do_not_wrap_if_one_line = true
ij_java_annotation_parameter_wrap = normal
ij_java_align_multiline_annotation_parameters = true
ij_java_class_count_to_use_import_on_demand = 100
ij_java_names_count_to_use_import_on_demand = 100
ij_java_imports_layout = $*, |, java.**, |, javax.**, |, org.**, |, com.**, |, *
ij_java_line_comment_at_first_column = false
ij_java_align_multiline_chained_methods = true
ij_java_align_multiline_parameters_in_calls = true
ij_java_align_multiline_binary_operation = true
ij_java_align_multiline_assignment = true
ij_java_align_multiline_ternary_operation = true
ij_java_align_multiline_throws_list = true
ij_java_align_multiline_extends_list = true
ij_java_align_multiline_array_initializer_expression = true
ij_java_call_parameters_wrap = normal
ij_java_method_parameters_wrap = normal
ij_java_resource_list_wrap = normal
ij_java_extends_list_wrap = normal
ij_java_throws_list_wrap = normal
ij_java_method_call_chain_wrap = normal
ij_java_binary_operation_wrap = normal
ij_java_ternary_operation_wrap = normal
ij_java_for_statement_wrap = normal
ij_java_array_initializer_wrap = normal
ij_java_assignment_wrap = normal
ij_java_assert_statement_wrap = normal
ij_java_if_brace_force = if_multiline
ij_java_do_while_brace_force = always
ij_java_while_brace_force = if_multiline
ij_java_for_brace_force = if_multiline
ij_java_wrap_long_lines = true
ij_java_parameter_annotation_wrap = normal
ij_java_enum_constants_wrap = split_into_lines
ij_java_keep_blank_lines_in_declarations = 1
ij_java_keep_blank_lines_in_code = 1
ij_java_keep_blank_lines_between_package_declaration_and_header = 1
ij_java_keep_blank_lines_before_right_brace = 1
ij_java_blank_lines_around_class = 1
ij_java_blank_lines_after_class_header = 1
ij_smart_tabs = true
[*.xml]
max_line_length = 120
ij_visual_guides = 120
ij_xml_text_wrap = off
ij_xml_space_inside_empty_tag = true
[.yaml]
ij_yaml_spaces_within_braces = false
ij_yaml_spaces_within_brackets = false
[.properties]
ij_properties_keep_blank_lines = true

8
.gitattributes vendored
View File

@ -5,12 +5,8 @@
apache-release.sh export-ignore
.licenserc.yaml export-ignore
.editorconfig export-ignore
hugegraph-store/hg-store-dist/src/assembly/static/bin/libjemalloc.so export-ignore
hugegraph-store/hg-store-dist/src/assembly/static/bin/libjemalloc_aarch64.so export-ignore
# ignored directory
.github/ export-ignore
.idea/ export-ignore
install-dist/scripts/ export-ignore
hugegraph-commons/hugegraph-dist/ export-ignore
docker/ export-ignore
hugegraph-dist/scripts/ export-ignore
#assembly/ export-ignore

View File

@ -14,7 +14,7 @@ jobs:
runs-on: ubuntu-latest
env:
USE_STAGE: 'true' # Whether to include the stage repository.
SCRIPT_DEPENDENCY: install-dist/scripts/dependency
SCRIPT_DEPENDENCY: hugegraph-server/hugegraph-dist/scripts/dependency
steps:
- name: Checkout source
uses: actions/checkout@v4
@ -32,7 +32,7 @@ jobs:
- name: mvn install
run: |
mvn install -Dmaven.test.skip=true -ntp
mvn install -DskipTests=true -ntp
- name: generate current dependencies
run: |
bash $SCRIPT_DEPENDENCY/regenerate_known_dependencies.sh current-dependencies.txt
@ -49,8 +49,7 @@ jobs:
uses: actions/dependency-review-action@v3
# Refer: https://github.com/actions/dependency-review-action
with:
# TODO: reset critical to low before releasing
fail-on-severity: critical
fail-on-severity: low
# Action will fail if dependencies don't match the list
#allow-licenses: Apache-2.0, MIT
#deny-licenses: GPL-3.0, AGPL-1.0, AGPL-3.0, LGPL-2.0, CC-BY-3.0

View File

@ -1,64 +0,0 @@
name: "HugeGraph-Commons CI"
on:
workflow_dispatch:
push:
branches:
- master
- /^release-.*$/
- /^test-.*$/
pull_request:
jobs:
build-commons:
runs-on: ubuntu-latest
env:
# TODO: reset use stage to false later
USE_STAGE: 'true' # Whether to include the stage repository.
strategy:
fail-fast: false
matrix:
JAVA_VERSION: ['11']
steps:
- name: Install JDK ${{ matrix.JAVA_VERSION }}
uses: actions/setup-java@v3
with:
java-version: ${{ matrix.JAVA_VERSION }}
distribution: 'zulu'
- name: Cache Maven packages
uses: actions/cache@v3
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 2
- name: Use staged maven repo settings
if: ${{ env.USE_STAGE == 'true' }}
run: |
cp $HOME/.m2/settings.xml /tmp/settings.xml
cp -vf .github/configs/settings.xml $HOME/.m2/settings.xml && cat $HOME/.m2/settings.xml
- name: Compile
run: |
mvn compile -Dmaven.javadoc.skip=true -ntp
- name: Run common test
run: |
mvn test -pl hugegraph-commons/hugegraph-common -Dtest=UnitTestSuite
- name: Run rpc test
run: |
mvn test -pl hugegraph-commons/hugegraph-rpc -Dtest=UnitTestSuite
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3.0.0
with:
file: target/jacoco.xml

View File

@ -1,4 +1,4 @@
name: "License Checker"
name: "License checker"
on:
push:

View File

@ -1,4 +1,4 @@
name: "HugeGraph-PD & Store & Hstore CI"
name: "hugegraph-pd-store-ci"
on:
push:
@ -14,8 +14,7 @@ jobs:
runs-on: ubuntu-latest
env:
# TODO: avoid duplicated env setup in pd & store
# TODO: reset use stage to false later
USE_STAGE: 'true' # Whether to include the stage repository.
USE_STAGE: 'false' # Whether to include the stage repository.
# TODO: remove outdated env
TRAVIS_DIR: hugegraph-server/hugegraph-dist/src/assembly/travis
REPORT_DIR: target/site/jacoco
@ -45,31 +44,30 @@ jobs:
cp $HOME/.m2/settings.xml /tmp/settings.xml
mv -vf .github/configs/settings.xml $HOME/.m2/settings.xml
- name: Run common test
run: |
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-common-test -DskipCommonsTests=true
- name: Run core test
run: |
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-core-test -DskipCommonsTests=true
# The above tests do not require starting a PD instance.
- name: Package
run: |
mvn clean package -U -Dmaven.javadoc.skip=true -Dmaven.test.skip=true -ntp
- name: Run common test
run: |
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-common-test
- name: Run core test
run: |
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-core-test
# The above tests do not require starting a PD instance.
- name: Prepare env and service
run: |
$TRAVIS_DIR/start-pd.sh
- name: Run client test
run: |
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-client-test -DskipCommonsTests=true
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-client-test
- name: Run rest test
run: |
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-rest-test -DskipCommonsTests=true
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-rest-test
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3.0.0
@ -80,7 +78,7 @@ jobs:
# TODO: avoid duplicated env setup
runs-on: ubuntu-latest
env:
USE_STAGE: 'true' # Whether to include the stage repository.
USE_STAGE: 'false' # Whether to include the stage repository.
# TODO: remove outdated env
TRAVIS_DIR: hugegraph-server/hugegraph-dist/src/assembly/travis
REPORT_DIR: target/site/jacoco
@ -119,29 +117,29 @@ jobs:
$TRAVIS_DIR/start-pd.sh
$TRAVIS_DIR/start-store.sh
- name: Run common test
run: |
mvn test -pl hugegraph-store/hg-store-test -am -P store-common-test -DskipCommonsTests=true
- name: Run client test
run: |
mvn test -pl hugegraph-store/hg-store-test -am -P store-client-test -DskipCommonsTests=true
mvn test -pl hugegraph-store/hg-store-test -am -P store-client-test -Dmaven.test.failure.ignore=true
- name: Run core test
run: |
mvn test -pl hugegraph-store/hg-store-test -am -P store-core-test -DskipCommonsTests=true
mvn test -pl hugegraph-store/hg-store-test -am -P store-core-test -Dmaven.test.failure.ignore=true
- name: Run common test
run: |
mvn test -pl hugegraph-store/hg-store-test -am -P store-common-test -Dmaven.test.failure.ignore=true
- name: Run rocksdb test
run: |
mvn test -pl hugegraph-store/hg-store-test -am -P store-rocksdb-test -DskipCommonsTests=true
mvn test -pl hugegraph-store/hg-store-test -am -P store-rocksdb-test -Dmaven.test.failure.ignore=true
- name: Run server test
run: |
mvn test -pl hugegraph-store/hg-store-test -am -P store-server-test -DskipCommonsTests=true
mvn test -pl hugegraph-store/hg-store-test -am -P store-server-test -Dmaven.test.failure.ignore=true
- name: Run raft-core test
- name: Run raftcore test
run: |
mvn test -pl hugegraph-store/hg-store-test -am -P store-raftcore-test -DskipCommonsTests=true
mvn test -pl hugegraph-store/hg-store-test -am -P store-raftcore-test -Dmaven.test.failure.ignore=true
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3.0.0

View File

@ -1,4 +1,4 @@
name: "HugeGraph-Server CI"
name: "hugegraph-server-ci"
on:
push:
@ -9,12 +9,11 @@ on:
pull_request:
jobs:
build-server:
build:
# TODO: we need test & replace it to ubuntu-24.04 or ubuntu-latest
runs-on: ubuntu-20.04
env:
# TODO: reset use stage to false later
USE_STAGE: 'true' # Whether to include the stage repository.
USE_STAGE: 'false' # Whether to include the stage repository.
TRAVIS_DIR: hugegraph-server/hugegraph-dist/src/assembly/travis
REPORT_DIR: target/site/jacoco
BACKEND: ${{ matrix.BACKEND }}

2
.gitignore vendored
View File

@ -57,7 +57,7 @@ build/
*.pyc
# maven ignore
apache-hugegraph-*/
apache-hugegraph-incubating-*/
output/
*.war
*.zip

View File

@ -91,15 +91,19 @@ header: # `header` section is configurations for source codes license header.
- '**/META-INF/MANIFEST.MF'
- '.repository/**'
- '**/.flattened-pom.xml'
# TODO: temporarily added to the ignore list, need handle them before releases ( ↓ )
- 'hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/id/SnowflakeIdGenerator.java'
- 'hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/traversal/optimize/HugeScriptTraversal.java'
- 'hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/type/Nameable.java'
- 'hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/type/define/Cardinality.java'
- 'hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/util/StringEncoding.java'
- 'hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/tinkerpop/ProcessBasicSuite.java'
- 'hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/tinkerpop/StructureBasicSuite.java'
- 'hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/opencypher/CypherOpProcessor.java'
- 'hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/opencypher/CypherPlugin.java'
- 'hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/metrics/ProcfsEntry.java'
- 'hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/metrics/ProcfsReader.java'
- 'hugegraph-store/hg-store-node/src/main/java/org/apache/hugegraph/store/node/metrics/ProcfsSmaps.java'
# TODO: temporarily added to the ignore list, need handle them before releases ( ↑ )
comment: on-failure # on what condition license-eye will comment on the pull request, `on-failure`, `always`, `never`.
# license-location-threshold specifies the index threshold where the license header can be located,

View File

@ -6,7 +6,7 @@ Required:
* Java 11
* Maven 3.5+
To build without executing tests: `mvn clean package -Dmaven.test.skip=true`
To build without executing tests: `mvn clean package -DskipTests`
## Building in IDEA

View File

@ -1,66 +0,0 @@
# hugegraph-commons
[![License](https://img.shields.io/badge/license-Apache%202-0E78BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)
[![codecov](https://codecov.io/gh/hugegraph/hugegraph-common/branch/master/graph/badge.svg)](https://codecov.io/gh/hugegraph/hugegraph-common)
[![Maven Central](https://maven-badges.herokuapp.com/maven-central/org.apache.hugegraph/hugegraph-common/badge.svg)](https://mvnrepository.com/artifact/org.apache.hugegraph/hugegraph-common)
[![CodeQL](https://github.com/apache/incubator-hugegraph-commons/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/apache/incubator-hugegraph-commons/actions/workflows/codeql-analysis.yml)
[![hugegraph-commons ci](https://github.com/apache/incubator-hugegraph-commons/actions/workflows/ci.yml/badge.svg)](https://github.com/apache/incubator-hugegraph-commons/actions/workflows/ci.yml)
hugegraph-commons is a common module for [HugeGraph](https://github.com/apache/hugegraph) and its peripheral components.
hugegraph-commons encapsulates locks, configurations, events, iterators, rest and some
numeric or collection util classes to simplify the development of HugeGraph and its components.
## Components
- Lock: atomic lock, key lock, lock group and lock manger
- Config: register and load config option with security check
- Event: listening and notification, do something asynchronously
- Iterator: some iterators with extra functions, map, filter, extend, etc.
- Rest: RESTful client implemented on OkHttp, POST, PUT, GET and DELETE
- Util: performance analyzer, version checker, numeric and Collection utils, log and exception utils, etc.
- Rpc: rpc component for inner module communication, currently it's based on [Sofa-RPC](https://github.com/sofastack/sofa-rpc)
You could use import the dependencies in `maven` like this:
```xml
<dependency>
<groupId>org.apache.hugegraph</groupId>
<artifactId>hugegraph-common</artifactId>
<version>1.2.0</version>
</dependency>
```
## Learn More
The [doc page](https://hugegraph.apache.org/docs/) contains more information about hugegraph modules.
And here are links of other repositories:
1. [hugegraph-server](https://github.com/apache/hugegraph) (graph's core component - OLTP server)
2. [hugegraph-toolchain](https://github.com/apache/hugegraph-toolchain) (include loader/dashboard/tool/client)
3. [hugegraph-computer](https://github.com/apache/hugegraph-computer) (graph processing system - OLAP)
4. [hugegraph-website/doc](https://github.com/apache/hugegraph-doc) (include doc & website code)
## Contributing
- Welcome to contribute to HugeGraph, please see [How to Contribute](https://hugegraph.apache.org/docs/contribution-guidelines/contribute/) for more information.
- Note: It's recommended to use [GitHub Desktop](https://desktop.github.com/) to greatly simplify the PR and commit process.
- Thank you to all the people who already contributed to HugeGraph!
[![contributors graph](https://contrib.rocks/image?repo=apache/hugegraph-commons)](https://github.com/apache/incubator-hugegraph-commons/graphs/contributors)
## Licence
Same as HugeGraph, hugegraph-commons are also licensed under [Apache 2.0](./LICENSE) License.
### Contact Us
---
- [GitHub Issues](https://github.com/apache/incubator-hugegraph-commons/issues): Feedback on usage issues and functional requirements (quick response)
- Feedback Email: [dev@hugegraph.apache.org](mailto:dev@hugegraph.apache.org) ([subscriber](https://hugegraph.apache.org/docs/contribution-guidelines/subscribe/) only)
- WeChat public account: Apache HugeGraph, welcome to scan this QR code to follow us.
<img src="https://raw.githubusercontent.com/apache/incubator-hugegraph-doc/master/assets/images/wechat.png" alt="QR png" width="350"/>

View File

@ -1,23 +0,0 @@
# hugegraph-common
[![License](https://img.shields.io/badge/license-Apache%202-0E78BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)
[![Build Status](https://travis-ci.org/hugegraph/hugegraph-common.svg?branch=master)](https://travis-ci.org/hugegraph/hugegraph-common)
[![codecov](https://codecov.io/gh/hugegraph/hugegraph-common/branch/master/graph/badge.svg)](https://codecov.io/gh/hugegraph/hugegraph-common)
[![Maven Central](https://maven-badges.herokuapp.com/maven-central/org.apache.hugegraph/hugegraph-common/badge.svg)](https://mvnrepository.com/artifact/org.apache.hugegraph/hugegraph-common)
hugegraph-common is a common module for [HugeGraph](https://github.com/hugegraph/hugegraph) and its peripheral components.
hugegraph-common encapsulates locks, configurations, events, iterators, rest and some
numeric or collection util classes to simplify the development of HugeGraph and
its components.
## Components
- Lock: atomic lock, key lock, lock group and lock manger
- Config: register and load config option with security check
- Event: listening and notification, do something asynchronously
- Iterator: some iterators with extra functions, map, filter, extend, etc.
- Rest: RESTful client implemented on Jersey, POST, PUT, GET and DELETE
- Util: Performance analyzer, version checker, numeric and Collection utils, log and exception utils, etc.
## Licence
The same as HugeGraph, hugegraph-common is also licensed under Apache 2.0 License.

View File

@ -1,302 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.hugegraph</groupId>
<artifactId>hugegraph-commons</artifactId>
<version>${revision}</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>hugegraph-common</artifactId>
<name>${project.artifactId}</name>
<url>https://github.com/apache/incubator-hugegraph-commons/tree/master/hugegraph-common</url>
<description>
hugegraph-common is a common module for HugeGraph and its peripheral components.
hugegraph-common encapsulates locks, configurations, events, iterators, rest and some
numeric or collection util classes to simplify the development of HugeGraph and its
components.
</description>
<properties>
<!-- Use parent params -->
<lombok.version>1.18.8</lombok.version>
<okhttp.version>4.10.0</okhttp.version>
</properties>
<dependencies>
<!-- Test -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>${mockito.version}</version>
<scope>test</scope>
</dependency>
<!-- Logging -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>${log4j2.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j2.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>${log4j2.version}</version>
</dependency>
<!-- Utility -->
<dependency>
<groupId>org.glassfish</groupId>
<artifactId>javax.json</artifactId>
<version>${javax.json.version}</version>
</dependency>
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>${commons.configuration.version}</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-configuration2</artifactId>
<version>${commons.configuration2.version}</version>
<exclusions>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
<exclusion>
<artifactId>commons-lang3</artifactId>
<groupId>org.apache.commons</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>${commons.lang3.version}</version>
</dependency>
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
<version>${commons.beanutils.version}</version>
<exclusions>
<exclusion>
<artifactId>commons-logging</artifactId>
<groupId>commons-logging</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>${commons.io.version}</version>
</dependency>
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
<version>${commons.collections.version}</version>
</dependency>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>${commons.codec.version}</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>${guava.version}</version>
<exclusions>
<exclusion>
<artifactId>jsr305</artifactId>
<groupId>com.google.code.findbugs</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
<version>${jsr305.version}</version>
</dependency>
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.10.8</version>
</dependency>
<!-- javassist -->
<dependency>
<groupId>org.javassist</groupId>
<artifactId>javassist</artifactId>
<version>${javassist.version}</version>
</dependency>
<!-- jackson -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-jaxb-annotations</artifactId>
<version>${jackson.version}</version>
<exclusions>
<exclusion>
<artifactId>jakarta.xml.bind-api</artifactId>
<groupId>jakarta.xml.bind</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.jaxrs</groupId>
<artifactId>jackson-jaxrs-base</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.jaxrs</groupId>
<artifactId>jackson-jaxrs-json-provider</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.sun.xml.bind</groupId>
<artifactId>jaxb-impl</artifactId>
<version>${sun.xml.version}</version>
<scope>runtime</scope>
<exclusions>
<exclusion>
<artifactId>jakarta.xml.bind-api</artifactId>
<groupId>jakarta.xml.bind</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>logging-interceptor</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>${lombok.version}</version>
<optional>true</optional>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp-bom</artifactId>
<version>${okhttp.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.2</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<outputDirectory>${project.parent.build.directory}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>apache-release</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

View File

@ -1,82 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.concurrent;
import java.util.concurrent.atomic.AtomicReference;
import org.apache.hugegraph.util.Log;
import org.slf4j.Logger;
public class AtomicLock {
private static final Logger LOG = Log.logger(LockManager.class);
private String name;
private AtomicReference<Thread> sign;
public AtomicLock(String name) {
this.name = name;
this.sign = new AtomicReference<>();
}
public boolean tryLock() {
Thread current = Thread.currentThread();
return this.sign.compareAndSet(null, current);
}
public void unlock() {
if (this.sign.get() == null) {
return;
}
Thread current = Thread.currentThread();
if (!this.sign.compareAndSet(current, null)) {
throw new RuntimeException(String.format(
"Thread '%s' trying to unlock '%s' " +
"which is held by other threads now.",
current.getName(), this.name));
}
}
public boolean lock(int retries) {
// The interval between retries is exponential growth, most wait
// interval is 2^(retries-1)s. If retries=0, don't retry.
if (retries < 0 || retries > 10) {
throw new IllegalArgumentException(String.format(
"Locking retry times should be in [0, 10], but got %d",
retries));
}
boolean isLocked = false;
try {
for (int i = 0; !(isLocked = this.tryLock()) && i < retries; i++) {
Thread.sleep(1000 * (1L << i));
}
} catch (InterruptedException ignored) {
LOG.info("Thread sleep is interrupted.");
}
return isLocked;
}
public String name() {
return this.name;
}
public void name(String name) {
this.name = name;
}
}

View File

@ -1,104 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.concurrent;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
import org.apache.hugegraph.util.E;
public class BarrierEvent {
private final Lock lock = new ReentrantLock();
private final Condition cond = lock.newCondition();
private volatile boolean signaled = false;
/**
* Wait forever until the signal is received.
* @throws InterruptedException if interrupted.
*/
public void await() throws InterruptedException {
this.lock.lock();
try {
while (!this.signaled) {
this.cond.await();
}
} finally {
this.lock.unlock();
}
}
/**
* Wait specified time in milliseconds.
* @param timeout: the time in millisecond to wait.
* @return true if signal is received, false if time out.
* @throws InterruptedException if interrupted.
*/
public boolean await(long timeout) throws InterruptedException {
E.checkArgument(timeout >= 0L,
"The time must be >= 0, but got '%d'.",
timeout);
long deadline = System.currentTimeMillis() + timeout;
this.lock.lock();
try {
while (!this.signaled) {
timeout = deadline - System.currentTimeMillis();
if (timeout > 0) {
this.cond.await(timeout, TimeUnit.MILLISECONDS);
}
if (System.currentTimeMillis() >= deadline) {
return this.signaled;
}
}
} finally {
this.lock.unlock();
}
return true;
}
public void reset() {
this.lock.lock();
try {
this.signaled = false;
} finally {
this.lock.unlock();
}
}
public void signal() {
this.lock.lock();
try {
this.signaled = true;
this.cond.signal();
} finally {
this.lock.unlock();
}
}
public void signalAll() {
this.lock.lock();
try {
this.signaled = true;
this.cond.signalAll();
} finally {
this.lock.unlock();
}
}
}

View File

@ -1,144 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.concurrent;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.locks.Lock;
import org.apache.hugegraph.util.E;
import com.google.common.collect.ImmutableList;
import com.google.common.util.concurrent.Striped;
/**
* KeyLock provide an interface of segment lock
*/
public class KeyLock {
private Striped<Lock> locks;
public KeyLock() {
// The default size is availableProcessors() * 4
this(Runtime.getRuntime().availableProcessors() << 2);
}
public KeyLock(int size) {
this.locks = Striped.lock(size);
}
private int indexOf(Lock lock) {
for (int i = 0; i < this.locks.size(); i++) {
if (this.locks.getAt(i) == lock) {
return i;
}
}
return -1;
}
/**
* Lock an object
* @param key The object to lock
* @return The lock(locked) of passed key
*/
public final Lock lock(Object key) {
E.checkArgument(key != null, "Lock key can't be null");
Lock lock = this.locks.get(key);
lock.lock();
return lock;
}
/**
* Unlock an object
* @param key The object to unlock
*/
public final void unlock(Object key) {
E.checkArgument(key != null, "Unlock key can't be null");
this.locks.get(key).unlock();
}
/**
* Lock a list of object with sorted order
* @param keys The objects to lock
* @return The locks(locked) of keys
*/
public final List<Lock> lockAll(Object... keys) {
E.checkArgument(keys != null && keys.length > 0,
"Lock keys can't be null or empty");
List<Lock> locks = new ArrayList<>(keys.length);
for (Object key : keys) {
E.checkArgument(key != null, "Lock key can't be null");
Lock lock = this.locks.get(key);
locks.add(lock);
}
locks.sort((a, b) -> {
int diff = a.hashCode() - b.hashCode();
if (diff == 0 && a != b) {
diff = this.indexOf(a) - this.indexOf(b);
assert diff != 0;
}
return diff;
});
for (Lock lock : locks) {
lock.lock();
}
return Collections.unmodifiableList(locks);
}
/**
* Lock two objects with sorted order
* NOTE: This is to optimize the performance of lockAll(keys)
* @param key1 The first object
* @param key2 The second object
* @return locks for the two objects
*/
public List<Lock> lockAll(Object key1, Object key2) {
E.checkArgument(key1 != null, "Lock key can't be null");
E.checkArgument(key2 != null, "Lock key can't be null");
Lock lock1 = this.locks.get(key1);
Lock lock2 = this.locks.get(key2);
int diff = lock1.hashCode() - lock2.hashCode();
if (diff == 0 && lock1 != lock2) {
diff = this.indexOf(lock1) - this.indexOf(lock2);
assert diff != 0;
}
List<Lock> locks = diff > 0 ?
ImmutableList.of(lock2, lock1) :
ImmutableList.of(lock1, lock2);
for (Lock lock : locks) {
lock.lock();
}
return locks;
}
/**
* Unlock a list of object
* @param locks The locks to unlock
*/
public final void unlockAll(List<Lock> locks) {
E.checkArgument(locks != null, "Unlock locks can't be null");
for (int i = locks.size(); i > 0; i--) {
assert this.indexOf(locks.get(i - 1)) != -1;
locks.get(i - 1).unlock();
}
}
}

View File

@ -1,85 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.concurrent;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
public class LockGroup {
private final String name;
private final Map<String, Object> locksMap;
public LockGroup(String lockGroup) {
this.name = lockGroup;
this.locksMap = new ConcurrentHashMap<>();
}
public Lock lock(String lockName) {
if (!this.locksMap.containsKey(lockName)) {
this.locksMap.putIfAbsent(lockName, new ReentrantLock());
}
return (Lock) this.locksMap.get(lockName);
}
public AtomicLock atomicLock(String lockName) {
if (!this.locksMap.containsKey(lockName)) {
this.locksMap.putIfAbsent(lockName, new AtomicLock(lockName));
}
return (AtomicLock) this.locksMap.get(lockName);
}
public ReadWriteLock readWriteLock(String lockName) {
if (!this.locksMap.containsKey(lockName)) {
this.locksMap.putIfAbsent(lockName, new ReentrantReadWriteLock());
}
return (ReadWriteLock) this.locksMap.get(lockName);
}
public KeyLock keyLock(String lockName) {
if (!this.locksMap.containsKey(lockName)) {
this.locksMap.putIfAbsent(lockName, new KeyLock());
}
return (KeyLock) this.locksMap.get(lockName);
}
public KeyLock keyLock(String lockName, int size) {
if (!this.locksMap.containsKey(lockName)) {
this.locksMap.putIfAbsent(lockName, new KeyLock(size));
}
return (KeyLock) this.locksMap.get(lockName);
}
public <K extends Comparable<K>> RowLock<K> rowLock(String lockName) {
if (!this.locksMap.containsKey(lockName)) {
this.locksMap.putIfAbsent(lockName, new RowLock<>());
}
Object value = this.locksMap.get(lockName);
@SuppressWarnings("unchecked")
RowLock<K> lock = (RowLock<K>) value;
return lock;
}
public String name() {
return this.name;
}
}

View File

@ -1,71 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.concurrent;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
public class LockManager {
private static final LockManager INSTANCE = new LockManager();
public static LockManager instance() {
return INSTANCE;
}
private Map<String, LockGroup> lockGroupMap;
private LockManager() {
this.lockGroupMap = new ConcurrentHashMap<>();
}
public boolean exists(String group) {
return this.lockGroupMap.containsKey(group);
}
public LockGroup create(String group) {
if (exists(group)) {
throw new RuntimeException(String.format(
"LockGroup '%s' already exists", group));
}
LockGroup lockGroup = new LockGroup(group);
LockGroup previous = this.lockGroupMap.putIfAbsent(group, lockGroup);
if (previous != null) {
return previous;
}
return lockGroup;
}
public LockGroup get(String group) {
LockGroup lockGroup = this.lockGroupMap.get(group);
if (lockGroup == null) {
throw new RuntimeException(String.format(
"LockGroup '%s' does not exists", group));
}
return lockGroup;
}
public void destroy(String group) {
if (this.exists(group)) {
this.lockGroupMap.remove(group);
} else {
throw new RuntimeException(String.format(
"LockGroup '%s' does not exists", group));
}
}
}

View File

@ -1,78 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.concurrent;
import java.util.List;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.ThreadFactory;
import org.apache.hugegraph.util.Log;
import org.slf4j.Logger;
public class PausableScheduledThreadPool extends ScheduledThreadPoolExecutor {
private static final Logger LOG = Log.logger(PausableScheduledThreadPool.class);
private volatile boolean paused = false;
public PausableScheduledThreadPool(int corePoolSize,
ThreadFactory factory) {
super(corePoolSize, factory);
}
public synchronized void pauseSchedule() {
this.paused = true;
LOG.info("PausableScheduledThreadPool was paused");
}
public synchronized void resumeSchedule() {
this.paused = false;
this.notifyAll();
LOG.info("PausableScheduledThreadPool was resumed");
}
@Override
protected void beforeExecute(Thread t, Runnable r) {
synchronized (this) {
while (this.paused) {
try {
this.wait();
} catch (InterruptedException e) {
LOG.warn("PausableScheduledThreadPool was interrupted");
}
}
}
super.beforeExecute(t, r);
}
@Override
public void shutdown() {
if (this.paused) {
this.resumeSchedule();
}
super.shutdown();
}
@Override
public List<Runnable> shutdownNow() {
if (this.paused) {
this.resumeSchedule();
}
return super.shutdownNow();
}
}

View File

@ -1,98 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.concurrent;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
import org.apache.hugegraph.util.E;
public class RowLock<K extends Comparable<K>> {
private final Map<K, Lock> locks = new ConcurrentHashMap<>();
private final ThreadLocal<Map<K, LocalLock>> localLocks =
ThreadLocal.withInitial(HashMap::new);
public void lock(K key) {
E.checkArgument(key != null, "Lock key can't be null");
LocalLock localLock = this.localLocks.get().get(key);
if (localLock != null) {
localLock.lockCount++;
} else {
Lock current = new ReentrantLock();
Lock previous = this.locks.putIfAbsent(key, current);
if (previous != null) {
current = previous;
}
current.lock();
this.localLocks.get().put(key, new LocalLock(current));
}
}
public void unlock(K key) {
E.checkArgument(key != null, "Unlock key can't be null");
LocalLock localLock = this.localLocks.get().get(key);
if (localLock == null) {
return;
}
if (--localLock.lockCount == 0) {
this.locks.remove(key, localLock.current);
this.localLocks.get().remove(key);
localLock.current.unlock();
}
E.checkState(localLock.lockCount >= 0,
"The lock count must be >= 0, but got %s",
localLock.lockCount);
}
public void lockAll(Set<K> keys) {
E.checkArgument(keys != null && keys.size() > 0,
"Lock keys can't be null or empty");
List<K> list = new ArrayList<>(keys);
Collections.sort(list);
for (K key : list) {
this.lock(key);
}
}
public void unlockAll(Set<K> keys) {
E.checkArgument(keys != null && keys.size() > 0,
"Unlock keys can't be null or empty");
for (K key : keys) {
this.unlock(key);
}
}
private static class LocalLock {
private final Lock current;
private int lockCount;
private LocalLock(Lock current) {
this.current = current;
this.lockCount = 1;
}
}
}

View File

@ -1,47 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.config;
import java.util.function.Function;
import org.apache.hugegraph.util.E;
import com.google.common.base.Predicate;
public class ConfigConvOption<T, R> extends TypedOption<T, R> {
private final Function<T, R> converter;
public ConfigConvOption(String name, String desc, Predicate<T> pred,
Function<T, R> convert, T value) {
this(name, false, desc, pred, convert, value);
}
@SuppressWarnings("unchecked")
public ConfigConvOption(String name, boolean required, String desc,
Predicate<T> pred, Function<T, R> convert,
T value) {
super(name, required, desc, pred, (Class<T>) value.getClass(), value);
E.checkNotNull(convert, "convert");
this.converter = convert;
}
@Override
public R convert(T value) {
return this.converter.apply(value);
}
}

View File

@ -1,39 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.config;
public class ConfigException extends RuntimeException {
private static final long serialVersionUID = -8711375282196157058L;
public ConfigException(String message) {
super(message);
}
public ConfigException(String message, Throwable cause) {
super(message, cause);
}
public ConfigException(String message, Object... args) {
super(String.format(message, args));
}
public ConfigException(String message, Throwable cause, Object... args) {
super(String.format(message, args), cause);
}
}

View File

@ -1,75 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.config;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.function.Function;
import org.apache.hugegraph.util.E;
import com.google.common.base.Predicate;
public class ConfigListConvOption<T, R> extends TypedOption<List<T>, List<R>> {
private final Class<T> elemClass;
private final Function<T, R> converter;
@SuppressWarnings("unchecked")
public ConfigListConvOption(String name, String desc,
Predicate<List<T>> pred, Function<T, R> convert,
T... values) {
this(name, false, desc, pred, convert, null, Arrays.asList(values));
}
@SuppressWarnings("unchecked")
public ConfigListConvOption(String name, boolean required, String desc,
Predicate<List<T>> pred, Function<T, R> convert,
Class<T> clazz, List<T> values) {
super(name, required, desc, pred,
(Class<List<T>>) values.getClass(), values);
E.checkNotNull(convert, "convert");
if (clazz == null && values.size() > 0) {
clazz = (Class<T>) values.get(0).getClass();
}
E.checkArgumentNotNull(clazz, "Element class can't be null");
this.elemClass = clazz;
this.converter = convert;
}
@Override
protected boolean forList() {
return true;
}
@Override
protected List<T> parse(String value) {
return ConfigListOption.convert(value, part -> {
return this.parse(part, this.elemClass);
});
}
@Override
public List<R> convert(List<T> values) {
List<R> results = new ArrayList<>(values.size());
for (T value : values) {
results.add(this.converter.apply(value));
}
return results;
}
}

View File

@ -1,79 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.config;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.function.Function;
import org.apache.hugegraph.util.E;
import com.google.common.base.Predicate;
public class ConfigListOption<T> extends ConfigOption<List<T>> {
private final Class<T> elemClass;
@SuppressWarnings("unchecked")
public ConfigListOption(String name, String desc,
Predicate<List<T>> pred, T... values) {
this(name, false, desc, pred, null, Arrays.asList(values));
}
@SuppressWarnings("unchecked")
public ConfigListOption(String name, boolean required, String desc,
Predicate<List<T>> pred, Class<T> clazz,
List<T> values) {
super(name, required, desc, pred,
(Class<List<T>>) values.getClass(), values);
if (clazz == null && values.size() > 0) {
clazz = (Class<T>) values.get(0).getClass();
}
E.checkArgumentNotNull(clazz, "Element class can't be null");
this.elemClass = clazz;
}
@Override
protected boolean forList() {
return true;
}
@Override
protected List<T> parse(String value) {
return convert(value, part -> this.parse(part, this.elemClass));
}
@SuppressWarnings("unchecked")
public static <T> List<T> convert(Object value, Function<String, ?> conv) {
if (value instanceof List) {
return (List<T>) value;
}
// If target data type is List, parse it as a list
String str = (String) value;
if (str.startsWith("[") && str.endsWith("]")) {
str = str.substring(1, str.length() - 1);
}
String[] parts = str.split(",");
List<T> results = new ArrayList<>(parts.length);
for (String part : parts) {
results.add((T) conv.apply(part.trim()));
}
return results;
}
}

View File

@ -1,37 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.config;
import com.google.common.base.Predicate;
public class ConfigOption<T> extends TypedOption<T, T> {
public ConfigOption(String name, String desc, T value) {
this(name, desc, null, value);
}
@SuppressWarnings("unchecked")
public ConfigOption(String name, String desc, Predicate<T> pred, T value) {
this(name, false, desc, pred, (Class<T>) value.getClass(), value);
}
public ConfigOption(String name, boolean required, String desc,
Predicate<T> pred, Class<T> type, T value) {
super(name, required, desc, pred, type, value);
}
}

View File

@ -1,216 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.config;
import java.io.File;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import javax.annotation.Nullable;
import org.apache.commons.configuration2.Configuration;
import org.apache.commons.configuration2.FileBasedConfiguration;
import org.apache.commons.configuration2.PropertiesConfiguration;
import org.apache.commons.configuration2.YAMLConfiguration;
import org.apache.commons.configuration2.builder.FileBasedConfigurationBuilder;
import org.apache.commons.configuration2.builder.fluent.Configurations;
import org.apache.commons.configuration2.builder.fluent.Parameters;
import org.apache.commons.configuration2.ex.ConfigurationException;
import org.apache.commons.configuration2.io.FileHandler;
import org.apache.commons.io.FilenameUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.Log;
import org.slf4j.Logger;
public class HugeConfig extends PropertiesConfiguration {
private static final Logger LOG = Log.logger(HugeConfig.class);
private String configPath;
public HugeConfig(Configuration config) {
loadConfig(config);
this.configPath = null;
}
public HugeConfig(String configFile) {
loadConfig(loadConfigFile(configFile));
this.configPath = configFile;
}
public HugeConfig(Map<String, Object> propertyMap) {
if (propertyMap == null) {
throw new ConfigException("The property map is null");
}
for (Map.Entry<String, Object> kv : propertyMap.entrySet()) {
this.addProperty(kv.getKey(), kv.getValue());
}
this.checkRequiredOptions();
}
private void loadConfig(Configuration config) {
if (config == null) {
throw new ConfigException("The config object is null");
}
this.setLayoutIfNeeded(config);
this.append(config);
this.checkRequiredOptions();
}
private void setLayoutIfNeeded(Configuration conf) {
if (!(conf instanceof PropertiesConfiguration)) {
return;
}
PropertiesConfiguration propConf = (PropertiesConfiguration) conf;
this.setLayout(propConf.getLayout());
}
@SuppressWarnings("unchecked")
public <T, R> R get(TypedOption<T, R> option) {
Object value = this.getProperty(option.name());
if (value == null) {
return option.defaultValue();
}
return (R) value;
}
public Map<String, String> getMap(ConfigListOption<String> option) {
List<String> values = this.get(option);
Map<String, String> result = new HashMap<>();
for (String value : values) {
String[] pair = value.split(":", 2);
E.checkState(pair.length == 2,
"Invalid option format for '%s': %s(expect KEY:VALUE)",
option.name(), value);
result.put(pair[0].trim(), pair[1].trim());
}
return result;
}
@Override
public void addPropertyDirect(String key, Object value) {
TypedOption<?, ?> option = OptionSpace.get(key);
if (option == null) {
LOG.warn("The config option '{}' is redundant, " +
"please ensure it has been registered", key);
} else {
// The input value is String(parsed by PropertiesConfiguration)
value = this.validateOption(key, value);
}
if (this.containsKey(key) && value instanceof List) {
for (Object item : (List<?>) value) {
super.addPropertyDirect(key, item);
}
} else {
super.addPropertyDirect(key, value);
}
}
@Override
protected void addPropertyInternal(String key, Object value) {
this.addPropertyDirect(key, value);
}
private Object validateOption(String key, Object value) {
TypedOption<?, ?> option = OptionSpace.get(key);
if (value instanceof String) {
return option.parseConvert((String) value);
}
Class<?> dataType = option.dataType();
if (dataType.isInstance(value)) {
return value;
}
throw new IllegalArgumentException(
String.format("Invalid value for key '%s': '%s'", key, value));
}
private void checkRequiredOptions() {
// TODO: Check required options must be contained in this map
}
public void save(File copiedFile) throws ConfigurationException {
FileHandler fileHandler = new FileHandler(this);
fileHandler.save(copiedFile);
}
@Nullable
public File file() {
if (StringUtils.isEmpty(this.configPath)) {
return null;
}
return new File(this.configPath);
}
public void file(String path) {
this.configPath = path;
}
private static Configuration loadConfigFile(String path) {
E.checkNotNull(path, "config path");
E.checkArgument(!path.isEmpty(),
"The config path can't be empty");
File file = new File(path);
return loadConfigFile(file);
}
private static Configuration loadConfigFile(File configFile) {
E.checkArgument(configFile.exists() &&
configFile.isFile() &&
configFile.canRead(),
"Please specify a proper config file rather than: '%s'",
configFile.toString());
try {
String fileName = configFile.getName();
String fileExtension = FilenameUtils.getExtension(fileName);
Configuration config;
Configurations configs = new Configurations();
switch (fileExtension) {
case "yml":
case "yaml":
Parameters params = new Parameters();
FileBasedConfigurationBuilder<FileBasedConfiguration> builder =
new FileBasedConfigurationBuilder(YAMLConfiguration.class)
.configure(params.fileBased().setFile(configFile));
config = builder.getConfiguration();
break;
case "xml":
config = configs.xml(configFile);
break;
default:
config = configs.properties(configFile);
break;
}
return config;
} catch (ConfigurationException e) {
throw new ConfigException("Unable to load config: '%s'",
e, configFile);
}
}
}

View File

@ -1,83 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.config;
import java.lang.reflect.Array;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import org.apache.commons.lang3.StringUtils;
import com.google.common.base.Predicate;
public final class OptionChecker {
public static <O> Predicate<O> disallowEmpty() {
return o -> {
if (o == null) {
return false;
}
if (o instanceof String) {
return StringUtils.isNotBlank((String) o);
}
if (o.getClass().isArray() && (Array.getLength(o) == 0)) {
return false;
}
return !(o instanceof Iterable) || ((Iterable<?>) o).iterator().hasNext();
};
}
@SuppressWarnings("unchecked")
public static <O> Predicate<O> allowValues(O... values) {
return o -> o != null && Arrays.asList(values).contains(o);
}
@SuppressWarnings("unchecked")
public static <O> Predicate<List<O>> inValues(O... values) {
return o -> o != null && new HashSet<>(Arrays.asList(values)).containsAll(o);
}
public static <N extends Number> Predicate<N> positiveInt() {
return number -> number != null && number.longValue() > 0;
}
public static <N extends Number> Predicate<N> nonNegativeInt() {
return number -> number != null && number.longValue() >= 0;
}
public static <N extends Number> Predicate<N> rangeInt(N min, N max) {
return number -> {
if (number == null) {
return false;
}
long value = number.longValue();
return value >= min.longValue() && value <= max.longValue();
};
}
public static <N extends Number> Predicate<N> rangeDouble(N min, N max) {
return number -> {
if (number == null) {
return false;
}
double value = number.doubleValue();
return value >= min.doubleValue() && value <= max.doubleValue();
};
}
}

View File

@ -1,59 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.config;
import java.lang.reflect.Field;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import org.apache.hugegraph.util.Log;
import org.slf4j.Logger;
public class OptionHolder {
private static final Logger LOG = Log.logger(HugeConfig.class);
protected Map<String, TypedOption<?, ?>> options;
public OptionHolder() {
this.options = new HashMap<>();
}
protected void registerOptions() {
for (Field field : this.getClass().getFields()) {
if (!TypedOption.class.isAssignableFrom(field.getType())) {
// Skip if not option
continue;
}
try {
TypedOption<?, ?> option = (TypedOption<?, ?>) field.get(this);
// Fields of subclass first, don't overwrite by superclass
this.options.putIfAbsent(option.name(), option);
} catch (Exception e) {
LOG.error("Failed to register option: {}", field, e);
throw new ConfigException(
"Failed to register option: %s", field);
}
}
}
public Map<String, TypedOption<?, ?>> options() {
return Collections.unmodifiableMap(this.options);
}
}

View File

@ -1,120 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.config;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.Collections;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.Log;
import org.slf4j.Logger;
public final class OptionSpace {
private static final Logger LOG = Log.logger(OptionSpace.class);
private static final Map<String, Class<? extends OptionHolder>> HOLDERS;
private static final Map<String, TypedOption<?, ?>> OPTIONS;
private static final String INSTANCE_METHOD = "instance";
static {
HOLDERS = new ConcurrentHashMap<>();
OPTIONS = new ConcurrentHashMap<>();
}
public static void register(String module, String holder) {
ClassLoader classLoader = OptionSpace.class.getClassLoader();
Class<?> clazz;
try {
clazz = classLoader.loadClass(holder);
} catch (ClassNotFoundException e) {
throw new ConfigException(
"Failed to load class of option holder '%s'", e, holder);
}
// Check subclass
if (!OptionHolder.class.isAssignableFrom(clazz)) {
throw new ConfigException(
"Class '%s' is not a subclass of OptionHolder", holder);
}
OptionHolder instance = null;
Exception exception = null;
try {
Method method = clazz.getMethod(INSTANCE_METHOD);
if (!Modifier.isStatic(method.getModifiers())) {
throw new NoSuchMethodException(INSTANCE_METHOD);
}
instance = (OptionHolder) method.invoke(null);
if (instance == null) {
exception = new ConfigException(
"Returned null from %s() method",
INSTANCE_METHOD);
}
} catch (NoSuchMethodException e) {
LOG.warn("Class {} does not has static method {}.",
holder, INSTANCE_METHOD);
exception = e;
} catch (InvocationTargetException e) {
LOG.warn("Can't call static method {} from class {}.",
INSTANCE_METHOD, holder);
exception = e;
} catch (IllegalAccessException e) {
LOG.warn("Illegal access while calling method {} from class {}.",
INSTANCE_METHOD, holder);
exception = e;
}
if (exception != null) {
throw new ConfigException("Failed to instantiate option holder: %s",
exception, holder);
}
register(module, instance);
}
public static void register(String module, OptionHolder holder) {
// Check exists
if (HOLDERS.containsKey(module)) {
LOG.warn("Already registered option holder: {} ({})",
module, HOLDERS.get(module));
}
E.checkArgumentNotNull(holder, "OptionHolder can't be null");
HOLDERS.put(module, holder.getClass());
OPTIONS.putAll(holder.options());
LOG.debug("Registered options for OptionHolder: {}",
holder.getClass().getSimpleName());
}
public static Set<String> keys() {
return Collections.unmodifiableSet(OPTIONS.keySet());
}
public static boolean containKey(String key) {
return OPTIONS.containsKey(key);
}
public static TypedOption<?, ?> get(String key) {
return OPTIONS.get(key);
}
}

View File

@ -1,194 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.config;
import java.lang.reflect.Method;
import java.util.List;
import java.util.Set;
import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.Log;
import org.apache.commons.configuration.PropertyConverter;
import org.slf4j.Logger;
import com.google.common.base.Joiner;
import com.google.common.base.Predicate;
import com.google.common.collect.ImmutableSet;
public class TypedOption<T, R> {
private static final Logger LOG = Log.logger(TypedOption.class);
private static final Set<Class<?>> ACCEPTED_DATA_TYPES;
private static final String ACCEPTED_DATA_TYPES_STRING;
static {
ACCEPTED_DATA_TYPES = ImmutableSet.of(
Boolean.class,
Short.class,
Integer.class,
Byte.class,
Long.class,
Float.class,
Double.class,
String.class,
String[].class,
Class.class,
List.class
);
ACCEPTED_DATA_TYPES_STRING = Joiner.on(", ").join(ACCEPTED_DATA_TYPES);
}
private final String name;
private final String desc;
private final boolean required;
private final Class<T> dataType;
private final T defaultValue;
private final Predicate<T> checkFunc;
@SuppressWarnings("unchecked")
public TypedOption(String name, boolean required, String desc,
Predicate<T> pred, Class<T> type, T value) {
E.checkNotNull(name, "name");
E.checkNotNull(type, "dataType");
this.name = name;
this.dataType = (Class<T>) this.checkAndAssignDataType(type);
this.defaultValue = value;
this.required = required;
this.desc = desc;
this.checkFunc = pred;
this.check(this.defaultValue);
}
private Class<?> checkAndAssignDataType(Class<T> dataType) {
for (Class<?> clazz : ACCEPTED_DATA_TYPES) {
if (clazz.isAssignableFrom(dataType)) {
return clazz;
}
}
String msg = String.format("Input data type '%s' doesn't belong " +
"to acceptable type set: [%s]",
dataType, ACCEPTED_DATA_TYPES_STRING);
throw new IllegalArgumentException(msg);
}
public String name() {
return this.name;
}
public Class<T> dataType() {
return this.dataType;
}
public String desc() {
return this.desc;
}
public boolean required() {
return this.required;
}
public R defaultValue() {
return this.convert(this.defaultValue);
}
public R parseConvert(String value) {
T parsed = this.parse(value);
this.check(parsed);
return this.convert(parsed);
}
@SuppressWarnings("unchecked")
protected T parse(String value) {
return (T) this.parse(value, this.dataType);
}
protected Object parse(String value, Class<?> dataType) {
if (dataType.equals(String.class)) {
return value;
} else if (dataType.equals(Class.class)) {
try {
if (value.startsWith("class")) {
value = value.substring("class".length()).trim();
}
return Class.forName(value);
} catch (ClassNotFoundException e) {
throw new ConfigException(
"Failed to parse Class from String '%s'", e, value);
}
} else if (List.class.isAssignableFrom(dataType)) {
E.checkState(this.forList(),
"List option can't be registered with class %s",
this.getClass().getSimpleName());
}
// Use PropertyConverter method `toXXX` convert value
String methodTo = "to" + dataType.getSimpleName();
try {
Method method = PropertyConverter.class.getMethod(
methodTo, Object.class);
return method.invoke(null, value);
} catch (ReflectiveOperationException e) {
LOG.error("Invalid type of value '{}' for option '{}'",
value, this.name, e);
throw new ConfigException(
"Invalid type of value '%s' for option '%s', " +
"expect '%s' type",
value, this.name, dataType.getSimpleName());
}
}
protected void check(Object value) {
E.checkNotNull(value, "value", this.name);
if (!this.dataType.isInstance(value)) {
throw new ConfigException(
"Invalid type of value '%s' for option '%s', " +
"expect type %s but got %s", value, this.name,
this.dataType.getSimpleName(),
value.getClass().getSimpleName());
}
if (this.checkFunc != null) {
@SuppressWarnings("unchecked")
T result = (T) value;
if (!this.checkFunc.apply(result)) {
throw new ConfigException("Invalid option value for '%s': %s",
this.name, value);
}
}
}
@SuppressWarnings("unchecked")
protected R convert(T value) {
return (R) value;
}
protected boolean forList() {
return false;
}
@Override
public String toString() {
return String.format("[%s]%s=%s", this.dataType.getSimpleName(),
this.name, this.defaultValue);
}
}

View File

@ -1,64 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.date;
import java.util.Date;
import java.util.TimeZone;
import org.joda.time.DateTimeZone;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
/**
* The SafeDateFormat actually is a proxy for joda DateTimeFormatter
*/
public class SafeDateFormat {
private static final int ONE_HOUR_MS = 3600 * 1000;
private final String pattern;
private DateTimeFormatter formatter;
public SafeDateFormat(String pattern) {
this.pattern = pattern;
this.formatter = DateTimeFormat.forPattern(pattern);
}
public synchronized void setTimeZone(String zoneId) {
int hoursOffset = TimeZone.getTimeZone(zoneId).getRawOffset() /
ONE_HOUR_MS;
DateTimeZone zone = DateTimeZone.forOffsetHours(hoursOffset);
this.formatter = this.formatter.withZone(zone);
}
public TimeZone getTimeZone() {
return this.formatter.getZone().toTimeZone();
}
public Date parse(String source) {
return this.formatter.parseDateTime(source).toDate();
}
public String format(Date date) {
return this.formatter.print(date.getTime());
}
public Object toPattern() {
return this.pattern;
}
}

View File

@ -1,71 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.event;
import java.util.Arrays;
import java.util.Collections;
import org.apache.hugegraph.util.E;
public class Event extends java.util.EventObject {
private static final long serialVersionUID = 1625973849208342813L;
private String name;
private Object[] args;
public Event(Object source, String event) {
this(source, event, Collections.emptyList().toArray());
}
public Event(Object source, String event, Object... args) {
super(source);
this.name = event;
this.args = args;
}
public String name() {
return this.name;
}
public Object[] args() {
return this.args;
}
public void checkArgs(Class<?>... classes) throws IllegalArgumentException {
E.checkArgument(this.args.length == classes.length,
"The args count of event '%s' should be %s(actual %s)",
this.name, classes.length, this.args.length);
int i = 0;
for (Class<?> c : classes) {
Object arg = this.args[i++];
if (arg == null) {
continue;
}
E.checkArgument(c.isAssignableFrom(arg.getClass()),
"The arg '%s'(%s) can't match %s",
arg, arg.getClass(), c);
}
}
@Override
public String toString() {
return String.format("Event{name='%s', args=%s}",
this.name, Arrays.asList(this.args));
}
}

View File

@ -1,197 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.event;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import javax.annotation.Nullable;
import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.ExecutorUtil;
import org.apache.hugegraph.util.Log;
import org.slf4j.Logger;
import org.apache.hugegraph.iterator.ExtendableIterator;
import com.google.common.collect.ImmutableList;
public class EventHub {
private static final Logger LOG = Log.logger(EventHub.class);
public static final String EVENT_WORKER = "event-worker-%d";
public static final String ANY_EVENT = "*";
private static final List<EventListener> EMPTY = ImmutableList.of();
// Event executor
private static ExecutorService executor = null;
private String name;
private Map<String, List<EventListener>> listeners;
public EventHub() {
this("hub");
}
public EventHub(String name) {
this(name, 1, Runtime.getRuntime().availableProcessors() << 2);
}
public EventHub(String name, int threadSize) {
LOG.debug("Create new EventHub {},threadSize {}", name, threadSize);
this.name = name;
this.listeners = new ConcurrentHashMap<>();
EventHub.init(threadSize);
}
public EventHub(String name, int corePoolSize, int maximumPoolSize) {
LOG.debug("Create new EventHub {},corePoolSize {}, maximumPoolSize {}", name, corePoolSize, maximumPoolSize);
this.name = name;
this.listeners = new ConcurrentHashMap<>();
EventHub.init(corePoolSize, maximumPoolSize);
}
public static synchronized void init(int poolSize) {
if (executor != null) {
return;
}
LOG.debug("Init pool(size {}) for EventHub", poolSize);
executor = ExecutorUtil.newFixedThreadPool(poolSize, EVENT_WORKER);
}
public static synchronized void init(int corePoolSize, int maximumPoolSize) {
LOG.debug("Init corePoolSize {}, maximumPoolSize {} for EventHub", corePoolSize, maximumPoolSize);
if (executor != null) {
LOG.debug("EventHub executor already initialized");
return;
}
executor = ExecutorUtil.newDynamicThreadExecutor(EVENT_WORKER, corePoolSize, maximumPoolSize);
}
public static synchronized boolean destroy(long timeout)
throws InterruptedException {
E.checkState(executor != null, "EventHub has not been initialized");
LOG.debug("Destroy pool for EventHub");
executor.shutdown();
return executor.awaitTermination(timeout, TimeUnit.SECONDS);
}
private static ExecutorService executor() {
ExecutorService e = executor;
E.checkState(e != null, "The event executor has been destroyed");
return e;
}
public String name() {
return this.name;
}
public boolean containsListener(String event) {
List<EventListener> ls = this.listeners.get(event);
return ls != null && ls.size() > 0;
}
public List<EventListener> listeners(String event) {
List<EventListener> ls = this.listeners.get(event);
return ls == null ? EMPTY : Collections.unmodifiableList(ls);
}
public void listen(String event, EventListener listener) {
E.checkNotNull(event, "event");
E.checkNotNull(listener, "event listener");
if (!this.listeners.containsKey(event)) {
this.listeners.putIfAbsent(event, new CopyOnWriteArrayList<>());
}
List<EventListener> ls = this.listeners.get(event);
assert ls != null : this.listeners;
ls.add(listener);
}
public List<EventListener> unlisten(String event) {
List<EventListener> ls = this.listeners.remove(event);
return ls == null ? EMPTY : Collections.unmodifiableList(ls);
}
public int unlisten(String event, EventListener listener) {
List<EventListener> ls = this.listeners.get(event);
if (ls == null) {
return 0;
}
int count = 0;
while (ls.remove(listener)) {
count++;
}
return count;
}
public Future<Integer> notify(String event, @Nullable Object... args) {
@SuppressWarnings("resource")
ExtendableIterator<EventListener> all = new ExtendableIterator<>();
List<EventListener> ls = this.listeners.get(event);
if (ls != null && !ls.isEmpty()) {
all.extend(ls.iterator());
}
List<EventListener> lsAny = this.listeners.get(ANY_EVENT);
if (lsAny != null && !lsAny.isEmpty()) {
all.extend(lsAny.iterator());
}
if (!all.hasNext()) {
return CompletableFuture.completedFuture(0);
}
Event ev = new Event(this, event, args);
// The submit will catch params: `all`(Listeners) and `ev`(Event)
return executor().submit(() -> {
int count = 0;
// Notify all listeners, and ignore the results
while (all.hasNext()) {
try {
all.next().event(ev);
count++;
} catch (Throwable e) {
LOG.warn("Failed to handle event: {}", ev, e);
}
}
return count;
});
}
public Object call(String event, @Nullable Object... args) {
List<EventListener> ls = this.listeners.get(event);
if (ls == null) {
throw new RuntimeException("Not found listener for: " + event);
} else if (ls.size() != 1) {
throw new RuntimeException("Too many listeners for: " + event);
}
EventListener listener = ls.get(0);
return listener.event(new Event(this, event, args));
}
}

View File

@ -1,27 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.event;
public interface EventListener extends java.util.EventListener {
/**
* The event callback
* @param event object
* @return event result
*/
Object event(Event event);
}

View File

@ -1,23 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.func;
public interface TriFunction<T1, T2, T3, R> {
R apply(T1 v1, T2 v2, T3 v3);
}

View File

@ -1,104 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
import java.util.Iterator;
import java.util.List;
import java.util.function.Function;
import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.InsertionOrderUtil;
import com.google.common.collect.ImmutableList;
public class BatchMapperIterator<T, R> extends WrappedIterator<R> {
private final int batch;
private final Iterator<T> originIterator;
private final Function<List<T>, Iterator<R>> mapperCallback;
private Iterator<R> batchIterator;
public BatchMapperIterator(int batch, Iterator<T> origin,
Function<List<T>, Iterator<R>> mapper) {
E.checkArgument(batch > 0, "Expect batch > 0, but got %s", batch);
this.batch = batch;
this.originIterator = origin;
this.mapperCallback = mapper;
this.batchIterator = null;
}
@Override
protected Iterator<T> originIterator() {
return this.originIterator;
}
@Override
protected final boolean fetch() {
if (this.batchIterator != null && this.fetchFromBatch()) {
return true;
}
List<T> batch = this.nextBatch();
assert this.batchIterator == null;
while (!batch.isEmpty()) {
// Do fetch
this.batchIterator = this.mapperCallback.apply(batch);
if (this.batchIterator != null && this.fetchFromBatch()) {
return true;
}
// Try next batch
batch = this.nextBatch();
}
return false;
}
protected final List<T> nextBatch() {
if (!this.originIterator.hasNext()) {
return ImmutableList.of();
}
List<T> list = InsertionOrderUtil.newList();
for (int i = 0; i < this.batch && this.originIterator.hasNext(); i++) {
T next = this.originIterator.next();
list.add(next);
}
return list;
}
protected final boolean fetchFromBatch() {
E.checkNotNull(this.batchIterator, "mapper results");
while (this.batchIterator.hasNext()) {
R result = this.batchIterator.next();
if (result != null) {
assert this.current == none();
this.current = result;
return true;
}
}
this.resetBatchIterator();
return false;
}
protected final void resetBatchIterator() {
if (this.batchIterator == null) {
return;
}
close(this.batchIterator);
this.batchIterator = null;
}
}

View File

@ -1,23 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
import java.util.Iterator;
public interface CIter<R> extends Iterator<R>, AutoCloseable, Metadatable {
}

View File

@ -1,98 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
import java.util.Deque;
import java.util.Iterator;
import java.util.concurrent.ConcurrentLinkedDeque;
import org.apache.hugegraph.util.E;
public class ExtendableIterator<T> extends WrappedIterator<T> {
private final Deque<Iterator<T>> itors;
private Iterator<T> currentIterator;
public ExtendableIterator() {
this.itors = new ConcurrentLinkedDeque<>();
this.currentIterator = null;
}
public ExtendableIterator(Iterator<T> iter) {
this();
this.extend(iter);
}
public ExtendableIterator(Iterator<T> itor1, Iterator<T> itor2) {
this();
this.extend(itor1);
this.extend(itor2);
}
public ExtendableIterator<T> extend(Iterator<T> iter) {
E.checkState(this.currentIterator == null,
"Can't extend iterator after iterating");
if (iter != null) {
this.itors.addLast(iter);
}
return this;
}
@Override
public void close() throws Exception {
for (Iterator<T> iter : this.itors) {
if (iter instanceof AutoCloseable) {
((AutoCloseable) iter).close();
}
}
}
@Override
protected Iterator<T> originIterator() {
return this.currentIterator;
}
@Override
protected boolean fetch() {
assert this.current == none();
if (this.itors.isEmpty()) {
return false;
}
if (this.currentIterator != null && this.currentIterator.hasNext()) {
this.current = this.currentIterator.next();
return true;
}
Iterator<T> first;
while ((first = this.itors.peekFirst()) != null && !first.hasNext()) {
if (first == this.itors.peekLast() && this.itors.size() == 1) {
this.currentIterator = first;
// The last one
return false;
}
close(this.itors.removeFirst());
}
assert first != null && first.hasNext();
this.currentIterator = first;
this.current = this.currentIterator.next();
return true;
}
}

View File

@ -1,51 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
import java.util.Iterator;
import java.util.function.Function;
public class FilterIterator<T> extends WrappedIterator<T> {
private final Iterator<T> originIterator;
private final Function<T, Boolean> filterCallback;
public FilterIterator(Iterator<T> origin, Function<T, Boolean> filter) {
this.originIterator = origin;
this.filterCallback = filter;
}
@Override
protected Iterator<T> originIterator() {
return this.originIterator;
}
@Override
protected final boolean fetch() {
while (this.originIterator.hasNext()) {
T next = this.originIterator.next();
// Do filter
if (next != null && this.filterCallback.apply(next)) {
assert this.current == none();
this.current = next;
return true;
}
}
return false;
}
}

View File

@ -1,50 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
import java.util.Iterator;
import java.util.function.Function;
import org.apache.hugegraph.util.E;
public class FlatMapperFilterIterator<T, R> extends FlatMapperIterator<T, R> {
private final Function<R, Boolean> filterCallback;
public FlatMapperFilterIterator(Iterator<T> origin,
Function<T, Iterator<R>> mapper,
Function<R, Boolean> filter) {
super(origin, mapper);
this.filterCallback = filter;
}
@Override
protected final boolean fetchFromBatch() {
E.checkNotNull(this.batchIterator, "mapper results");
while (this.batchIterator.hasNext()) {
R result = this.batchIterator.next();
if (result != null && this.filterCallback.apply(result)) {
assert this.current == none();
this.current = result;
return true;
}
}
this.resetBatchIterator();
return false;
}
}

View File

@ -1,89 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
import java.util.Iterator;
import java.util.function.Function;
import org.apache.hugegraph.util.E;
public class FlatMapperIterator<T, R> extends WrappedIterator<R> {
private final Iterator<T> originIterator;
private final Function<T, Iterator<R>> mapperCallback;
protected Iterator<R> batchIterator;
public FlatMapperIterator(Iterator<T> origin,
Function<T, Iterator<R>> mapper) {
this.originIterator = origin;
this.mapperCallback = mapper;
this.batchIterator = null;
}
@Override
public void close() throws Exception {
this.resetBatchIterator();
super.close();
}
@Override
protected Iterator<T> originIterator() {
return this.originIterator;
}
@Override
protected final boolean fetch() {
if (this.batchIterator != null && this.fetchFromBatch()) {
return true;
}
while (this.originIterator.hasNext()) {
T next = this.originIterator.next();
assert this.batchIterator == null;
// Do fetch
this.batchIterator = this.mapperCallback.apply(next);
if (this.batchIterator != null && this.fetchFromBatch()) {
return true;
}
}
return false;
}
protected boolean fetchFromBatch() {
E.checkNotNull(this.batchIterator, "mapper results");
while (this.batchIterator.hasNext()) {
R result = this.batchIterator.next();
if (result != null) {
assert this.current == none();
this.current = result;
return true;
}
}
this.resetBatchIterator();
return false;
}
protected final void resetBatchIterator() {
if (this.batchIterator == null) {
return;
}
close(this.batchIterator);
this.batchIterator = null;
}
}

View File

@ -1,64 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
import java.util.Iterator;
import java.util.function.Function;
public class LimitIterator<T> extends WrappedIterator<T> {
private final Iterator<T> originIterator;
private final Function<T, Boolean> filterCallback;
public LimitIterator(Iterator<T> origin, Function<T, Boolean> filter) {
this.originIterator = origin;
this.filterCallback = filter;
}
@Override
protected Iterator<T> originIterator() {
return this.originIterator;
}
@Override
protected final boolean fetch() {
while (this.originIterator.hasNext()) {
T next = this.originIterator.next();
if (next == null) {
continue;
}
// Do filter
boolean reachLimit = this.filterCallback.apply(next);
if (reachLimit) {
this.closeOriginIterator();
return false;
}
assert this.current == none();
this.current = next;
return true;
}
return false;
}
protected final void closeOriginIterator() {
if (this.originIterator == null) {
return;
}
close(this.originIterator);
}
}

View File

@ -1,78 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
import java.util.Collection;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import org.apache.hugegraph.util.InsertionOrderUtil;
public class ListIterator<T> extends WrappedIterator<T> {
private final Iterator<T> originIterator;
private final Iterator<T> resultsIterator;
private final Collection<T> results;
public ListIterator(long capacity, Iterator<T> origin) {
List<T> results = InsertionOrderUtil.newList();
while (origin.hasNext()) {
if (capacity >= 0L && results.size() >= capacity) {
throw new IllegalArgumentException(
"The iterator exceeded capacity " + capacity);
}
results.add(origin.next());
}
this.originIterator = origin;
this.results = Collections.unmodifiableList(results);
this.resultsIterator = this.results.iterator();
}
public ListIterator(Collection<T> origin) {
this.originIterator = origin.iterator();
this.results = origin instanceof List ?
Collections.unmodifiableList((List<T>) origin) :
Collections.unmodifiableCollection(origin);
this.resultsIterator = this.results.iterator();
}
@Override
public void remove() {
this.resultsIterator.remove();
}
public Collection<T> list() {
return this.results;
}
@Override
protected boolean fetch() {
assert this.current == none();
if (!this.resultsIterator.hasNext()) {
return false;
}
this.current = this.resultsIterator.next();
return true;
}
@Override
protected Iterator<T> originIterator() {
return this.originIterator;
}
}

View File

@ -1,51 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
import java.util.Iterator;
import java.util.function.Function;
public class MapperIterator<T, R> extends WrappedIterator<R> {
private final Iterator<T> originIterator;
private final Function<T, R> mapperCallback;
public MapperIterator(Iterator<T> origin, Function<T, R> mapper) {
this.originIterator = origin;
this.mapperCallback = mapper;
}
@Override
protected Iterator<T> originIterator() {
return this.originIterator;
}
@Override
protected final boolean fetch() {
while (this.originIterator.hasNext()) {
T next = this.originIterator.next();
R result = this.mapperCallback.apply(next);
if (result != null) {
assert this.current == none();
this.current = result;
return true;
}
}
return false;
}
}

View File

@ -1,23 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
public interface Metadatable {
Object metadata(String meta, Object... args);
}

View File

@ -1,99 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.iterator;
import java.util.Iterator;
import java.util.NoSuchElementException;
public abstract class WrappedIterator<R> implements CIter<R> {
private static final Object NONE = new Object();
protected R current;
public WrappedIterator() {
this.current = none();
}
@Override
public boolean hasNext() {
if (this.current != none()) {
return true;
}
return this.fetch();
}
@Override
public R next() {
if (this.current == none()) {
this.fetch();
if (this.current == none()) {
throw new NoSuchElementException();
}
}
R current = this.current;
this.current = none();
return current;
}
@Override
public void remove() {
Iterator<?> iterator = this.originIterator();
if (iterator == null) {
throw new NoSuchElementException(
"The origin iterator can't be null for removing");
}
iterator.remove();
}
@Override
public void close() throws Exception {
Iterator<?> iterator = this.originIterator();
if (iterator instanceof AutoCloseable) {
((AutoCloseable) iterator).close();
}
}
@Override
public Object metadata(String meta, Object... args) {
Iterator<?> iterator = this.originIterator();
if (iterator instanceof Metadatable) {
return ((Metadatable) iterator).metadata(meta, args);
}
throw new IllegalStateException("Original iterator is not Metadatable");
}
@SuppressWarnings("unchecked")
protected static final <R> R none() {
return (R) NONE;
}
public static void close(Iterator<?> iterator) {
if (iterator instanceof AutoCloseable) {
try {
((AutoCloseable) iterator).close();
} catch (Exception e) {
throw new IllegalStateException("Failed to close iterator");
}
}
}
protected abstract Iterator<?> originIterator();
protected abstract boolean fetch();
}

View File

@ -1,106 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.license;
import java.util.Date;
import java.util.List;
import org.apache.commons.lang3.time.DateUtils;
import com.fasterxml.jackson.annotation.JsonFormat;
import com.fasterxml.jackson.annotation.JsonProperty;
public class LicenseCommonParam {
@JsonProperty("subject")
private String subject;
@JsonProperty("issued_time")
@JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss", timezone = "GMT+8")
private Date issuedTime = new Date();
@JsonProperty("not_before")
@JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss", timezone = "GMT+8")
private Date notBefore = this.issuedTime;
@JsonProperty("not_after")
@JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss", timezone = "GMT+8")
private Date notAfter = DateUtils.addDays(this.notBefore, 30);
@JsonProperty("consumer_type")
private String consumerType = "user";
@JsonProperty("consumer_amount")
private Integer consumerAmount = 1;
@JsonProperty("description")
private String description = "";
@JsonProperty("extra_params")
private List<LicenseExtraParam> extraParams;
public LicenseCommonParam() {
// pass
}
public LicenseCommonParam(String subject, String description,
Date issued, Date notBefore, Date notAfter,
String consumerType, int consumerAmount,
List<LicenseExtraParam> extraParams) {
this.subject = subject;
this.description = description;
this.issuedTime = issued;
this.notBefore = notBefore;
this.notAfter = notAfter;
this.consumerType = consumerType;
this.consumerAmount = consumerAmount;
this.extraParams = extraParams;
}
public String subject() {
return this.subject;
}
public Date issuedTime() {
return this.issuedTime;
}
public Date notBefore() {
return this.notBefore;
}
public Date notAfter() {
return this.notAfter;
}
public String consumerType() {
return this.consumerType;
}
public Integer consumerAmount() {
return this.consumerAmount;
}
public String description() {
return this.description;
}
public List<LicenseExtraParam> extraParams() {
return this.extraParams;
}
}

View File

@ -1,61 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.license;
import com.fasterxml.jackson.annotation.JsonAlias;
import com.fasterxml.jackson.annotation.JsonProperty;
public class LicenseCreateParam extends LicenseCommonParam {
@JsonProperty("private_alias")
private String privateAlias;
@JsonAlias("key_ticket")
@JsonProperty("key_password")
private String keyPassword;
@JsonAlias("store_ticket")
@JsonProperty("store_password")
private String storePassword;
@JsonProperty("privatekey_path")
private String privateKeyPath;
@JsonProperty("license_path")
private String licensePath;
public String privateAlias() {
return this.privateAlias;
}
public String keyPassword() {
return this.keyPassword;
}
public String storePassword() {
return this.storePassword;
}
public String privateKeyPath() {
return this.privateKeyPath;
}
public String licensePath() {
return this.licensePath;
}
}

View File

@ -1,119 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.license;
import com.fasterxml.jackson.annotation.JsonProperty;
public class LicenseExtraParam {
public static final int NO_LIMIT = -1;
@JsonProperty("id")
private String id;
@JsonProperty("version")
private String version;
@JsonProperty("graphs")
private int graphs;
@JsonProperty("ip")
private String ip;
@JsonProperty("mac")
private String mac;
@JsonProperty("cpus")
private int cpus;
// The unit is MB
@JsonProperty("ram")
private int ram;
@JsonProperty("threads")
private int threads;
// The unit is MB
@JsonProperty("memory")
private int memory;
@JsonProperty("nodes")
private int nodes;
// The unit is MB
@JsonProperty("data_size")
private long dataSize;
@JsonProperty("vertices")
private long vertices;
@JsonProperty("edges")
private long edges;
public String id() {
return this.id;
}
public String version() {
return this.version;
}
public int graphs() {
return this.graphs;
}
public String ip() {
return this.ip;
}
public String mac() {
return this.mac;
}
public int cpus() {
return this.cpus;
}
public int ram() {
return this.ram;
}
public int threads() {
return this.threads;
}
public int memory() {
return this.memory;
}
public int nodes() {
return this.nodes;
}
public long dataSize() {
return this.dataSize;
}
public long vertices() {
return this.vertices;
}
public long edges() {
return this.edges;
}
}

View File

@ -1,60 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.license;
import com.fasterxml.jackson.annotation.JsonAlias;
import com.fasterxml.jackson.annotation.JsonProperty;
public class LicenseInstallParam {
@JsonProperty("subject")
private String subject;
@JsonProperty("public_alias")
private String publicAlias;
@JsonAlias("store_ticket")
@JsonProperty("store_password")
private String storePassword;
@JsonProperty("publickey_path")
private String publicKeyPath;
@JsonProperty("license_path")
private String licensePath;
public String subject() {
return this.subject;
}
public String publicAlias() {
return this.publicAlias;
}
public String storePassword() {
return this.storePassword;
}
public String licensePath() {
return this.licensePath;
}
public String publicKeyPath() {
return this.publicKeyPath;
}
}

View File

@ -1,32 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.license;
public interface LicenseManager {
LicenseParams installLicense() throws Exception;
void uninstallLicense() throws Exception;
LicenseParams verifyLicense() throws Exception;
interface VerifyCallback {
void onVerifyLicense(LicenseParams params) throws Exception;
}
}

View File

@ -1,30 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.license;
import org.apache.commons.lang.NotImplementedException;
import org.apache.hugegraph.license.LicenseManager.VerifyCallback;
public class LicenseManagerFactory {
public static LicenseManager create(LicenseInstallParam param,
VerifyCallback veryfyCallback) {
throw new NotImplementedException("No LicenseManager available");
}
}

View File

@ -1,45 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.license;
import java.util.Date;
import java.util.List;
public class LicenseParams extends LicenseCommonParam {
public LicenseParams() {
super();
}
public LicenseParams(String subject, String description,
Date issued, Date notBefore, Date notAfter,
String consumerType, int consumerAmount,
List<LicenseExtraParam> extraParams) {
super(subject, description, issued, notBefore, notAfter,
consumerType, consumerAmount, extraParams);
}
public LicenseExtraParam matchParam(String id) {
for (LicenseExtraParam param : this.extraParams()) {
if (param.id().equals(id)) {
return param;
}
}
return null;
}
}

View File

@ -1,126 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.license;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.net.SocketException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
public class MachineInfo {
private List<String> ipAddressList;
private List<String> macAddressList;
public MachineInfo() {
this.ipAddressList = null;
this.macAddressList = null;
}
public List<String> getIpAddress() {
if (this.ipAddressList != null) {
return this.ipAddressList;
}
this.ipAddressList = new ArrayList<>();
List<InetAddress> inetAddresses = this.getLocalAllInetAddress();
if (inetAddresses != null && !inetAddresses.isEmpty()) {
this.ipAddressList = inetAddresses.stream()
.map(InetAddress::getHostAddress)
.distinct()
.map(String::toLowerCase)
.collect(Collectors.toList());
}
return this.ipAddressList;
}
public List<String> getMacAddress() {
if (this.macAddressList != null) {
return this.macAddressList;
}
this.macAddressList = new ArrayList<>();
List<InetAddress> inetAddresses = this.getLocalAllInetAddress();
if (inetAddresses != null && !inetAddresses.isEmpty()) {
// Get the Mac address of all network interfaces
List<String> list = new ArrayList<>();
Set<String> uniqueValues = new HashSet<>();
for (InetAddress inetAddress : inetAddresses) {
String macByInetAddress = this.getMacByInetAddress(inetAddress);
if (uniqueValues.add(macByInetAddress)) {
list.add(macByInetAddress);
}
}
this.macAddressList = list;
}
return this.macAddressList;
}
public List<InetAddress> getLocalAllInetAddress() {
Enumeration<NetworkInterface> interfaces;
try {
interfaces = NetworkInterface.getNetworkInterfaces();
} catch (SocketException e) {
throw new RuntimeException("Failed to get network interfaces");
}
List<InetAddress> result = new ArrayList<>();
while (interfaces.hasMoreElements()) {
NetworkInterface nw = interfaces.nextElement();
for (Enumeration<InetAddress> inetAddresses = nw.getInetAddresses();
inetAddresses.hasMoreElements(); ) {
InetAddress inetAddr = inetAddresses.nextElement();
if (!inetAddr.isLoopbackAddress() &&
!inetAddr.isLinkLocalAddress() &&
!inetAddr.isMulticastAddress()) {
result.add(inetAddr);
}
}
}
return result;
}
public String getMacByInetAddress(InetAddress inetAddr) {
byte[] mac;
try {
mac = NetworkInterface.getByInetAddress(inetAddr)
.getHardwareAddress();
} catch (Exception e) {
throw new RuntimeException(String.format(
"Failed to get mac address for inet address '%s'",
inetAddr));
}
StringBuilder sb = new StringBuilder();
for (int i = 0; i < mac.length; i++) {
if (i != 0) {
sb.append("-");
}
String temp = Integer.toHexString(mac[i] & 0xff);
if (temp.length() == 1) {
sb.append("0").append(temp);
} else {
sb.append(temp);
}
}
return sb.toString().toUpperCase();
}
}

View File

@ -1,185 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.perf;
import java.util.List;
import org.apache.hugegraph.perf.PerfUtil.FastMap;
public final class LightStopwatch implements Stopwatch {
private long lastStartTime = -1L;
private long times = 0L;
private long totalCost = 0L;
private long totalChildrenTimes = -1L;
private final String name;
private final Path parent;
private final Path id;
private final FastMap<String, Stopwatch> children;
public LightStopwatch(String name, Stopwatch parent) {
this(name, parent.id());
parent.child(name, this);
}
public LightStopwatch(String name, Path parent) {
this.name = name;
this.parent = parent;
this.id = Stopwatch.id(parent, name);
this.children = new FastMap<>();
}
@Override
public Path id() {
return this.id;
}
@Override
public String name() {
return this.name;
}
@Override
public Path parent() {
return this.parent;
}
@Override
public void lastStartTime(long startTime) {
this.lastStartTime = startTime;
}
@Override
public void startTime(long startTime) {
this.times++;
this.lastStartTime = startTime;
}
@Override
public void endTime(long startTime) {
this.totalCost += PerfUtil.now() - this.lastStartTime;
}
@Override
public long times() {
return this.times;
}
@Override
public long totalTimes() {
if (this.totalChildrenTimes > 0L) {
return this.times + this.totalChildrenTimes;
}
return this.times;
}
@Override
public long totalChildrenTimes() {
return this.totalChildrenTimes;
}
@Override
public long totalCost() {
return this.totalCost;
}
@Override
public void totalCost(long totalCost) {
this.totalCost = totalCost;
}
@Override
public long minCost() {
return -1L;
}
@Override
public long maxCost() {
return -1L;
}
@Override
public long totalWasted() {
return 0L;
}
@Override
public long totalSelfWasted() {
return 0L;
}
@Override
public long totalChildrenWasted() {
return -1L;
}
@Override
public void fillChildrenTotal(List<Stopwatch> children) {
// Fill total times of children
this.totalChildrenTimes = children.stream().mapToLong(Stopwatch::totalTimes).sum();
}
@Override
public LightStopwatch copy() {
try {
return (LightStopwatch) super.clone();
} catch (CloneNotSupportedException e) {
throw new RuntimeException(e);
}
}
@Override
public Stopwatch child(String name) {
return this.children.get(name);
}
@Override
public Stopwatch child(String name, Stopwatch watch) {
if (watch == null) {
return this.children.remove(name);
}
return this.children.put(name, watch);
}
@Override
public boolean empty() {
return this.children.size() == 0;
}
@Override
public void clear() {
this.lastStartTime = -1L;
this.times = 0L;
this.totalCost = 0L;
this.totalChildrenTimes = -1L;
this.children.clear();
}
@Override
public String toString() {
return String.format("{parent:%s,name:%s," +
"times:%s,totalChildrenTimes:%s,totalCost:%s}",
this.parent, this.name,
this.times, this.totalChildrenTimes,
this.totalCost);
}
}

View File

@ -1,302 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.perf;
import java.util.List;
import java.util.function.BiFunction;
import org.apache.hugegraph.util.Log;
import org.slf4j.Logger;
import org.apache.hugegraph.testutil.Whitebox;
public final class NormalStopwatch implements Stopwatch {
private static final Logger LOG = Log.logger(Stopwatch.class);
private static final String MULTI_THREAD_ACCESS_ERROR =
"There may be multi-threaded access, ensure " +
"not call PerfUtil.profileSingleThread(true) when " +
"multithreading.";
private long lastStartTime = -1L;
private long times = 0L;
private long totalCost = 0L;
private long minCost = Long.MAX_VALUE;
private long maxCost = 0L;
private long totalSelfWasted = 0L;
private long totalChildrenWasted = -1L;
private long totalChildrenTimes = -1L;
private final String name;
private final Path parent;
private final Path id;
private final PerfUtil.FastMap<String, Stopwatch> children;
public NormalStopwatch(String name, Stopwatch parent) {
this(name, parent.id());
parent.child(name, this);
}
public NormalStopwatch(String name, Path parent) {
this.name = name;
this.parent = parent;
this.id = Stopwatch.id(parent, name);
this.children = new PerfUtil.FastMap<>();
}
@Override
public Path id() {
return this.id;
}
@Override
public String name() {
return this.name;
}
@Override
public Path parent() {
return this.parent;
}
@Override
public void lastStartTime(long startTime) {
this.lastStartTime = startTime;
}
@Override
public void startTime(long startTime) {
assert this.lastStartTime == -1L : MULTI_THREAD_ACCESS_ERROR;
this.times++;
this.lastStartTime = startTime;
long endTime = PerfUtil.now();
long wastedTime = endTime - startTime;
if (wastedTime <= 0L) {
wastedTime += eachStartWastedLost;
}
this.totalSelfWasted += wastedTime;
}
@Override
public void endTime(long startTime) {
assert startTime >= this.lastStartTime && this.lastStartTime != -1L :
MULTI_THREAD_ACCESS_ERROR;
long endTime = PerfUtil.now();
// The following code cost about 3ns~4ns
long wastedTime = endTime - startTime;
if (wastedTime <= 0L) {
wastedTime += eachEndWastedLost;
}
long cost = endTime - this.lastStartTime;
if (this.minCost > cost) {
this.minCost = cost;
}
if (this.maxCost < cost) {
this.maxCost = cost;
}
this.totalCost += cost;
this.totalSelfWasted += wastedTime;
this.lastStartTime = -1L;
}
@Override
public long times() {
return this.times;
}
@Override
public long totalCost() {
return this.totalCost;
}
@Override
public void totalCost(long totalCost) {
this.totalCost = totalCost;
}
@Override
public long minCost() {
return this.minCost;
}
@Override
public long maxCost() {
return this.maxCost;
}
@Override
public long totalTimes() {
if (this.totalChildrenTimes > 0L) {
return this.times + this.totalChildrenTimes;
}
return this.times;
}
@Override
public long totalChildrenTimes() {
return this.totalChildrenTimes;
}
@Override
public long totalWasted() {
if (this.totalChildrenWasted > 0L) {
return this.totalSelfWasted + this.totalChildrenWasted;
}
return this.totalSelfWasted;
}
@Override
public long totalSelfWasted() {
return this.totalSelfWasted;
}
@Override
public long totalChildrenWasted() {
return this.totalChildrenWasted;
}
@Override
public void fillChildrenTotal(List<Stopwatch> children) {
// Fill total wasted cost of children
this.totalChildrenWasted = children.stream().mapToLong(Stopwatch::totalWasted).sum();
// Fill total times of children
this.totalChildrenTimes = children.stream().mapToLong(Stopwatch::totalTimes).sum();
}
@Override
public Stopwatch copy() {
try {
return (Stopwatch) super.clone();
} catch (CloneNotSupportedException e) {
throw new RuntimeException(e);
}
}
@Override
public Stopwatch child(String name) {
return this.children.get(name);
}
@Override
public Stopwatch child(String name, Stopwatch watch) {
if (watch == null) {
return this.children.remove(name);
}
return this.children.put(name, watch);
}
@Override
public boolean empty() {
return this.children.size() == 0;
}
@Override
public void clear() {
this.lastStartTime = -1L;
this.times = 0L;
this.totalCost = 0L;
this.minCost = Long.MAX_VALUE;
this.maxCost = 0L;
this.totalSelfWasted = 0L;
this.totalChildrenWasted = -1L;
this.totalChildrenTimes = -1L;
this.children.clear();
}
@Override
public String toString() {
return String.format("{parent:%s,name:%s," +
"times:%s,totalChildrenTimes:%s," +
"totalCost:%s,minCost:%s,maxCost:%s," +
"totalSelfWasted:%s,totalChildrenWasted:%s}",
this.parent, this.name,
this.times, this.totalChildrenTimes,
this.totalCost, this.minCost, this.maxCost,
this.totalSelfWasted, this.totalChildrenWasted);
}
private static long eachStartWastedLost = 0L;
private static long eachEndWastedLost = 0L;
protected static void initEachWastedLost() {
int times = 100000000;
PerfUtil.LocalStack<Stopwatch> callStack = Whitebox.getInternalState(
PerfUtil.instance(), "callStack");
long baseStart = PerfUtil.now();
for (int i = 0; i < times; i++) {
PerfUtil.instance();
}
long baseCost = PerfUtil.now() - baseStart;
BiFunction<String, Runnable, Long> testEachCost = (name, test) -> {
long start = PerfUtil.now();
test.run();
long end = PerfUtil.now();
long cost = end - start - baseCost;
if (cost < 0L) {
cost = 0L;
}
long eachCost = cost / times;
LOG.info("Wasted time test: cost={}ms, base_cost={}ms, {}={}ns",
cost / 1000000.0, baseCost / 1000000.0, name, eachCost);
return eachCost;
};
String startName = "each_start_cost";
eachStartWastedLost = testEachCost.apply(startName, () -> {
Stopwatch watch = PerfUtil.instance().start(startName);
PerfUtil.instance().end(startName);
for (int i = 0; i < times; i++) {
// Test call start()
PerfUtil.instance().start(startName);
// Mock end()
watch.lastStartTime(-1L);
callStack.pop();
}
});
String endName = "each_end_cost";
eachEndWastedLost = testEachCost.apply(endName, () -> {
Stopwatch watch = PerfUtil.instance().start(endName);
PerfUtil.instance().end(endName);
for (int i = 0; i < times; i++) {
// Mock start()
callStack.push(watch);
watch.lastStartTime(0L);
// Test call start()
PerfUtil.instance().end(endName);
watch.totalCost(0L);
}
});
}
}

View File

@ -1,687 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.perf;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.EmptyStackException;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.CountDownLatch;
import java.util.function.BiConsumer;
import java.util.function.Function;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.slf4j.Logger;
import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.Log;
import org.apache.hugegraph.util.ReflectionUtil;
import org.apache.hugegraph.func.TriFunction;
import org.apache.hugegraph.testutil.Assert.ThrowableConsumer;
import org.apache.hugegraph.perf.Stopwatch.Path;
import com.google.common.reflect.ClassPath.ClassInfo;
import javassist.CannotCompileException;
import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtMethod;
public final class PerfUtil {
private static final Logger LOG = Log.logger(PerfUtil.class);
private static final int DEFAULT_CAPACITY = 1024;
private static final ThreadLocal<PerfUtil> INSTANCE = new ThreadLocal<>();
private static PerfUtil SINGLE_INSTANCE = null;
private static Thread SINGLE_THREAD = null;
private static LocalTimer LOCAL_TIMER = null;
private static boolean LIGHT_WATCH = false;
private final Map<Path, Stopwatch> stopwatches;
private final LocalStack<Stopwatch> callStack;
private final Stopwatch root;
private PerfUtil() {
this.stopwatches = new HashMap<>(DEFAULT_CAPACITY);
this.callStack = new LocalStack<>(DEFAULT_CAPACITY);
this.root = newStopwatch(Path.ROOT_NAME, Path.EMPTY);
}
public static PerfUtil instance() {
if (SINGLE_INSTANCE != null &&
SINGLE_THREAD == Thread.currentThread()) {
// Return the only one instance for single thread, for performance
return SINGLE_INSTANCE;
}
PerfUtil p = INSTANCE.get();
if (p == null) {
p = new PerfUtil();
INSTANCE.set(p);
}
return p;
}
public static void profileSingleThread(boolean yes) {
SINGLE_INSTANCE = yes ? PerfUtil.instance() : null;
SINGLE_THREAD = yes ? Thread.currentThread() : null;
}
public static void useLocalTimer(boolean yes) {
if (yes) {
if (LOCAL_TIMER != null) {
return;
}
LOCAL_TIMER = new LocalTimer();
try {
LOCAL_TIMER.startTimeUpdateLoop();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
if (!LIGHT_WATCH) {
NormalStopwatch.initEachWastedLost();
}
} else {
if (LOCAL_TIMER == null) {
return;
}
try {
LOCAL_TIMER.stop();
} catch (InterruptedException e) {
throw new RuntimeException(e);
} finally {
LOCAL_TIMER = null;
}
}
}
public static void useLightStopwatch(boolean yes) {
if (yes != LIGHT_WATCH) {
PerfUtil instance = INSTANCE.get();
boolean empty = instance == null || instance.empty();
String message = "Please call clear() before switching " +
"light-stopwatch due to there is dirty watch";
E.checkArgument(empty, message);
}
LIGHT_WATCH = yes;
}
protected static long now() {
if (LOCAL_TIMER != null) {
return LOCAL_TIMER.now();
}
// System.nanoTime() cost about 40 ns each call
return System.nanoTime();
}
protected static Stopwatch newStopwatch(String name, Path parent) {
return LIGHT_WATCH ? new LightStopwatch(name, parent) :
new NormalStopwatch(name, parent);
}
protected static Stopwatch newStopwatch(String name, Stopwatch parent) {
return LIGHT_WATCH ? new LightStopwatch(name, parent) :
new NormalStopwatch(name, parent);
}
public Stopwatch start(String name) {
long start = now();
Stopwatch parent = this.callStack.empty() ?
this.root : this.callStack.peek();
// Get watch by name from local tree
Stopwatch watch = parent.child(name);
if (watch == null) {
watch = newStopwatch(name, parent);
assert !this.stopwatches.containsKey(watch.id()) : watch;
this.stopwatches.put(watch.id(), watch);
}
this.callStack.push(watch);
watch.startTime(start);
return watch;
}
public Stopwatch start2(String name) {
long start = now(); // cost 70 ns with System.nanoTime()
Path parent = this.callStack.empty() ?
Path.EMPTY : this.callStack.peek().id();
Path id = Stopwatch.id(parent, name); // cost 130
// Get watch by id from global map
Stopwatch watch = this.stopwatches.get(id); // cost 170
if (watch == null) {
watch = newStopwatch(name, parent);
this.stopwatches.put(watch.id(), watch); // cost 180
}
this.callStack.push(watch); // cost 190
watch.startTime(start);
return watch;
}
public void end(String name) {
long start = LIGHT_WATCH ? 0L : now();
Stopwatch watch = this.callStack.pop();
if (watch == null || watch.name() != name) {
throw new IllegalArgumentException("Invalid watch name: " + name);
}
watch.endTime(start);
}
public boolean empty() {
return this.stopwatches.isEmpty() && this.root.empty();
}
public void clear() {
String error = "Can't be cleared when the call has not ended yet";
E.checkState(this.callStack.empty(), error);
this.stopwatches.clear();
this.root.clear();
}
public void profilePackage(String... packages) throws Throwable {
Set<String> loadedClasses = new HashSet<>();
Function<String, Boolean> inPackage = (cls) -> {
for (String pkg : packages) {
if (cls.startsWith(pkg)) {
return true;
}
}
return false;
};
ThrowableConsumer<String> profileClassIfPresent = (cls) -> {
if (!loadedClasses.contains(cls)) {
// Profile super class
for (String s : ReflectionUtil.superClasses(cls)) {
if (!loadedClasses.contains(s) && inPackage.apply(s)) {
profileClass(s);
loadedClasses.add(s);
}
}
// Profile self class
profileClass(cls);
loadedClasses.add(cls);
}
};
Iterator<ClassInfo> classes = ReflectionUtil.classes(packages);
while (classes.hasNext()) {
String cls = classes.next().getName();
// Profile self class
profileClassIfPresent.accept(cls);
// Profile nested class
for (String s : ReflectionUtil.nestedClasses(cls)) {
profileClassIfPresent.accept(s);
}
}
}
public void profileClass(String... classes) throws Throwable {
ClassPool classPool = ClassPool.getDefault();
for (String cls : classes) {
CtClass ctClass = classPool.get(cls);
List<CtMethod> methods = ReflectionUtil.getMethodsAnnotatedWith(
ctClass, Watched.class, false);
for (CtMethod method : methods) {
profile(method);
}
// Load class and make it effective
if (!methods.isEmpty()) {
ctClass.toClass();
}
}
}
private void profile(CtMethod ctMethod)
throws CannotCompileException, ClassNotFoundException {
final String START =
"org.apache.hugegraph.perf.PerfUtil.instance().start(\"%s\");";
final String END =
"org.apache.hugegraph.perf.PerfUtil.instance().end(\"%s\");";
Watched annotation = (Watched) ctMethod.getAnnotation(Watched.class);
String name = annotation.value();
if (name.isEmpty()) {
name = ctMethod.getName();
}
if (!annotation.prefix().isEmpty()) {
name = annotation.prefix() + "." + name;
}
ctMethod.insertBefore(String.format(START, name));
// Insert as a finally-statement
ctMethod.insertAfter(String.format(END, name), true);
LOG.debug("Profiled for: '{}' [{}]", name, ctMethod.getLongName());
}
@Override
public String toString() {
return this.stopwatches.toString();
}
public String toJson() {
StringBuilder sb = new StringBuilder(8 + this.stopwatches.size() * 96);
sb.append('{');
for (Map.Entry<Path, Stopwatch> w : this.stopwatches.entrySet()) {
sb.append('"');
sb.append(w.getKey());
sb.append('"');
sb.append(':');
sb.append(w.getValue().toJson());
sb.append(',');
}
if (!this.stopwatches.isEmpty()) {
sb.deleteCharAt(sb.length() - 1);
}
sb.append('}');
return sb.toString();
}
// TODO: move toECharts() method out of this class
public String toECharts() {
TriFunction<Integer, Integer, List<Stopwatch>, String> formatLevel = (
totalDepth, depth, items) -> {
float factor = 100.0f / (totalDepth + 1);
float showFactor = 1 + (totalDepth - depth) / (float) depth;
float radiusFrom = depth * factor;
float radiusTo = depth * factor + factor;
if (depth == 1) {
radiusFrom = 0;
}
StringBuilder sb = new StringBuilder(8 + items.size() * 128);
sb.append('{');
sb.append("name: 'Total Cost',");
sb.append("type: 'pie',");
sb.append(String.format("radius: ['%s%%', '%s%%'],",
radiusFrom, radiusTo));
sb.append(String.format(
"label: {normal: {position: 'inner', formatter:" +
"function(params) {" +
" if (params.percent > %s) return params.data.name;" +
" else return '';" +
"}}},", showFactor));
sb.append("data: [");
items.sort((i, j) -> i.id().compareTo(j.id()));
for (Stopwatch w : items) {
sb.append('{');
sb.append("id:'");
sb.append(w.id());
sb.append("',");
sb.append("name:'");
sb.append(w.name());
sb.append("',");
sb.append("value:");
// w.totalCost() - w.totalWasted() ?
sb.append(w.totalCost());
sb.append(',');
sb.append("cost:");
sb.append(w.totalCost() / 1000000.0);
sb.append(',');
sb.append("minCost:");
sb.append(w.minCost());
sb.append(',');
sb.append("maxCost:");
sb.append(w.maxCost());
sb.append(',');
sb.append("wasted:");
sb.append(w.totalWasted() / 1000000.0);
sb.append(',');
sb.append("selfWasted:");
sb.append(w.totalSelfWasted() / 1000000.0);
sb.append(',');
sb.append("times:");
sb.append(w.times());
sb.append(',');
sb.append("totalTimes:");
sb.append(w.totalTimes());
sb.append('}');
sb.append(',');
}
if (!items.isEmpty()) {
sb.deleteCharAt(sb.length() - 1);
}
sb.append("]}");
return sb.toString();
};
BiConsumer<List<Stopwatch>, List<Stopwatch>> fillChildrenTotal =
(itemsOfLn, itemsOfLnParent) -> {
for (Stopwatch parent : itemsOfLnParent) {
List<Stopwatch> children = itemsOfLn.stream().filter(c -> {
return c.parent().equals(parent.id());
}).collect(Collectors.toList());
parent.fillChildrenTotal(children);
}
};
BiConsumer<List<Stopwatch>, List<Stopwatch>> fillOther =
(itemsOfLn, itemsOfLnParent) -> {
for (Stopwatch parent : itemsOfLnParent) {
Stream<Stopwatch> children = itemsOfLn.stream().filter(c -> {
return c.parent().equals(parent.id());
});
// Fill other cost
long sumCost = children.mapToLong(Stopwatch::totalCost).sum();
long otherCost = parent.totalCost() - sumCost;
if (otherCost > 0L) {
Stopwatch other = newStopwatch("~", parent.id());
other.totalCost(otherCost);
itemsOfLn.add(other);
}
}
};
Map<Path, Stopwatch> items = this.stopwatches;
Map<Integer, List<Stopwatch>> levelItems = new HashMap<>();
int maxDepth = 0;
for (Map.Entry<Path, Stopwatch> e : items.entrySet()) {
int depth = e.getKey().toString().split("/").length;
List<Stopwatch> levelItem = levelItems.get(depth);
if (levelItem == null) {
levelItem = new LinkedList<>();
levelItems.putIfAbsent(depth, levelItem);
}
levelItem.add(e.getValue().copy());
if (depth > maxDepth) {
maxDepth = depth;
}
}
// Fill wasted cost from the outermost to innermost
for (int i = maxDepth; i > 0; i--) {
assert levelItems.containsKey(i) : i;
List<Stopwatch> itemsOfI = levelItems.get(i);
List<Stopwatch> itemsOfParent = levelItems.get(i - 1);
if (itemsOfParent != null) {
// Fill total value of children
fillChildrenTotal.accept(itemsOfI, itemsOfParent);
}
}
StringBuilder sb = new StringBuilder(8 + items.size() * 128);
// Output results header
sb.append("{");
sb.append("tooltip: {trigger: 'item', " +
"formatter: function(params) {" +
" return params.data.name + ' ' + params.percent + '% <br/>'" +
" + 'cost: ' + params.data.cost + ' (ms) <br/>'" +
" + 'min cost: ' + params.data.minCost + ' (ns) <br/>'" +
" + 'max cost: ' + params.data.maxCost + ' (ns) <br/>'" +
" + 'wasted: ' + params.data.wasted + ' (ms) <br/>'" +
" + 'self wasted: ' + params.data.selfWasted + ' (ms) <br/>'" +
" + 'times: ' + params.data.times + '<br/>'" +
" + 'total times: ' + params.data.totalTimes + '<br/>'" +
" + 'path: ' + params.data.id + '<br/>';" +
"}");
sb.append("},");
sb.append("series: [");
// Output results data
for (int i = 1; i <= maxDepth; i++) {
assert levelItems.containsKey(i) : i;
List<Stopwatch> itemsOfI = levelItems.get(i);
List<Stopwatch> itemsOfParent = levelItems.get(i - 1);
if (itemsOfParent != null) {
// Fill other cost for non-root level, ignore root level (i=1)
fillOther.accept(itemsOfI, itemsOfParent);
}
// Output items of level I
sb.append(formatLevel.apply(maxDepth, i, itemsOfI));
sb.append(',');
}
if (!items.isEmpty()) {
sb.deleteCharAt(sb.length() - 1);
}
sb.append("]}");
return sb.toString();
}
public static final class LocalTimer {
// Header: 4 bytes classptr + 8 bytes markword
private volatile long padding11 = 0L;
private volatile long padding12 = 0L;
private volatile long padding13 = 0L;
private volatile long padding14 = 0L;
private volatile long padding15 = 0L;
private volatile long padding16 = 0L; // the 1st 64 bytes
private volatile long time = 0L;
private volatile long padding21 = 0L;
private volatile long padding22 = 0L;
private volatile long padding23 = 0L;
private volatile long padding24 = 0L;
private volatile long padding25 = 0L;
private volatile long padding26 = 0L;
private volatile long padding27 = 0L; // the 2nd 64 bytes
private volatile boolean running = false;
private Thread thread = null;
public long now() {
// Read current ns time (be called frequently)
return this.time;
}
public void startTimeUpdateLoop() throws InterruptedException {
assert this.thread == null;
assert this.preventOptimizePadding() == 0L;
this.running = true;
CountDownLatch started = new CountDownLatch(1);
this.thread = new Thread(() -> {
started.countDown();
while (this.running) {
this.time = System.nanoTime();
// Prevent frequent updates for perf (5.2s => 3.6s for 8kw)
Thread.yield();
}
}, "LocalTimer");
this.thread.setDaemon(true);
this.thread.start();
started.await();
}
public void stop() throws InterruptedException {
this.running = false;
if (this.thread != null) {
this.thread.join();
}
}
public long preventOptimizePadding() {
long p1 = this.padding11 + this.padding12 + this.padding13 +
this.padding14 + this.padding15 + this.padding16;
long p2 = this.padding21 + this.padding22 + this.padding23 +
this.padding24 + this.padding25 + this.padding26 +
this.padding27;
return p1 + p2;
}
}
public static final class LocalStack<T> {
private final Object[] elementData;
private int elementCount;
public LocalStack(int capacity) {
this.elementData = new Object[capacity];
this.elementCount = 0;
}
int size() {
return this.elementCount;
}
boolean empty() {
return this.elementCount == 0;
}
public void push(T elem) {
this.elementData[this.elementCount++] = elem;
}
public T pop() {
if (this.elementCount == 0) {
throw new EmptyStackException();
}
this.elementCount--;
@SuppressWarnings("unchecked")
T elem = (T) this.elementData[this.elementCount];
this.elementData[this.elementCount] = null;
return elem;
}
public T peek() {
if (this.elementCount == 0) {
throw new EmptyStackException();
}
@SuppressWarnings("unchecked")
T elem = (T) this.elementData[this.elementCount - 1];
return elem;
}
}
public static final class FastMap<K, V> {
private final Map<K, V> hashMap;
private K key1;
private K key2;
private K key3;
private V val1;
private V val2;
private V val3;
public FastMap() {
this.hashMap = new HashMap<>();
}
public int size() {
return this.hashMap.size();
}
public boolean containsKey(Object key) {
return this.hashMap.containsKey(key);
}
public V get(Object key) {
if (key == this.key1) {
return this.val1;
} else if (key == this.key2) {
return this.val2;
} else if (key == this.key3) {
return this.val3;
}
return this.hashMap.get(key);
}
public V put(K key, V value) {
if (this.key1 == null) {
this.key1 = key;
this.val1 = value;
} else if (this.key2 == null) {
this.key2 = key;
this.val2 = value;
} else if (this.key3 == null) {
this.key3 = key;
this.val3 = value;
}
return this.hashMap.put(key, value);
}
public V remove(Object key) {
if (key == this.key1) {
this.key1 = null;
this.val1 = null;
} else if (key == this.key2) {
this.key2 = null;
this.val2 = null;
} else if (key == this.key3) {
this.key3 = null;
this.val3 = null;
}
return this.hashMap.remove(key);
}
public void clear() {
this.key1 = null;
this.key2 = null;
this.key3 = null;
this.val1 = null;
this.val2 = null;
this.val3 = null;
this.hashMap.clear();
}
}
@Retention(RetentionPolicy.RUNTIME)
@Target({ ElementType.METHOD, ElementType.CONSTRUCTOR })
public @interface Watched {
String value() default "";
String prefix() default "";
}
}

View File

@ -1,148 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.perf;
import java.util.List;
public interface Stopwatch extends Cloneable {
Path id();
String name();
Path parent();
void startTime(long startTime);
void endTime(long startTime);
void lastStartTime(long startTime);
long times();
long totalTimes();
long totalChildrenTimes();
long totalCost();
void totalCost(long otherCost);
long minCost();
long maxCost();
long totalWasted();
long totalSelfWasted();
long totalChildrenWasted();
void fillChildrenTotal(List<Stopwatch> children);
Stopwatch copy();
Stopwatch child(String name);
Stopwatch child(String name, Stopwatch watch);
boolean empty();
void clear();
default String toJson() {
int len = 200 + this.name().length() + this.parent().length();
StringBuilder sb = new StringBuilder(len);
sb.append("{");
sb.append("\"parent\":\"").append(this.parent()).append("\"");
sb.append(",\"name\":\"").append(this.name()).append("\"");
sb.append(",\"times\":").append(this.times());
sb.append(",\"total_cost\":").append(this.totalCost());
sb.append(",\"min_cost\":").append(this.minCost());
sb.append(",\"max_cost\":").append(this.maxCost());
sb.append(",\"total_self_wasted\":").append(this.totalSelfWasted());
sb.append(",\"total_children_wasted\":").append(
this.totalChildrenWasted());
sb.append(",\"total_children_times\":").append(
this.totalChildrenTimes());
sb.append("}");
return sb.toString();
}
static Path id(Path parent, String name) {
if (parent == Path.EMPTY && name == Path.ROOT_NAME) {
return Path.EMPTY;
}
return new Path(parent, name);
}
final class Path implements Comparable<Path> {
public static final String ROOT_NAME = "root";
public static final Path EMPTY = new Path("");
private final String path;
public Path(String self) {
this.path = self;
}
public Path(Path parent, String name) {
if (parent == EMPTY) {
this.path = name;
} else {
this.path = parent.path + '/' + name;
}
}
public int length() {
return this.path.length();
}
@Override
public int hashCode() {
return this.path.hashCode();
}
@Override
public boolean equals(Object obj) {
if (this.hashCode() != obj.hashCode()) {
return false;
}
if (!(obj instanceof Path)) {
return false;
}
Path other = (Path) obj;
return this.path.equals(other.path);
}
@Override
public int compareTo(Path other) {
return this.path.compareTo(other.path);
}
@Override
public String toString() {
return this.path;
}
public boolean endsWith(String name) {
return this.path.endsWith(name);
}
}
}

View File

@ -1,490 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.rest;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.security.KeyStore;
import java.util.Arrays;
import java.util.Collection;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.TimeUnit;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.SSLSocketFactory;
import javax.net.ssl.TrustManager;
import javax.net.ssl.TrustManagerFactory;
import javax.net.ssl.X509TrustManager;
import org.apache.commons.lang3.StringUtils;
import org.apache.hugegraph.util.JsonUtilCommon;
import org.jetbrains.annotations.NotNull;
import com.google.common.collect.ImmutableMap;
import lombok.SneakyThrows;
import okhttp3.ConnectionPool;
import okhttp3.HttpUrl;
import okhttp3.MediaType;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
import okio.BufferedSink;
import okio.GzipSink;
import okio.Okio;
/**
* This class provides an abstract implementation of the RestClient interface.
* It provides methods for making HTTP requests (GET, POST, PUT, DELETE) to a REST API.
* Note: It uses the OkHttp library to make these requests for now.
*/
public abstract class AbstractRestClient implements RestClient {
private final ThreadLocal<String> authContext;
private final OkHttpClient client;
private final String baseUrl;
public AbstractRestClient(String url, int timeout) {
this(url, RestClientConfig.builder().timeout(timeout).build());
}
public AbstractRestClient(String url, String user, String password, int timeout) {
this(url, RestClientConfig.builder()
.user(user)
.password(password)
.timeout(timeout)
.build());
}
public AbstractRestClient(String url, int timeout, int idleTime,
int maxConns, int maxConnsPerRoute) {
this(url, RestClientConfig.builder()
.idleTime(idleTime)
.timeout(timeout)
.maxConns(maxConns)
.maxConnsPerRoute(maxConnsPerRoute)
.build());
}
public AbstractRestClient(String url, String user, String password, int timeout,
int maxConns, int maxConnsPerRoute,
String trustStoreFile, String trustStorePassword) {
this(url, RestClientConfig.builder()
.user(user).password(password)
.timeout(timeout)
.maxConns(maxConns)
.maxConnsPerRoute(maxConnsPerRoute)
.trustStoreFile(trustStoreFile)
.trustStorePassword(trustStorePassword)
.build());
}
public AbstractRestClient(String url, String token, int timeout,
int maxConns, int maxConnsPerRoute,
String trustStoreFile, String trustStorePassword) {
this(url, RestClientConfig.builder()
.token(token)
.timeout(timeout)
.maxConns(maxConns)
.maxConnsPerRoute(maxConnsPerRoute)
.trustStoreFile(trustStoreFile)
.trustStorePassword(trustStorePassword)
.build());
}
public AbstractRestClient(String url, RestClientConfig config) {
this.baseUrl = url;
this.client = buildOkHttpClient(config);
this.authContext = new InheritableThreadLocal<>();
}
private static RequestBody buildRequestBody(Object body, RestHeaders headers) {
String contentType = parseContentType(headers);
String bodyContent;
if (RestHeaders.APPLICATION_JSON.equals(contentType)) {
if (body == null) {
bodyContent = "{}";
} else if (body instanceof String) {
bodyContent = (String) body;
} else {
bodyContent = JsonUtilCommon.toJson(body);
}
} else {
bodyContent = String.valueOf(body);
}
RequestBody requestBody = RequestBody.create(bodyContent.getBytes(),
MediaType.parse(contentType));
if (headers != null &&
"gzip".equals(headers.get(RestHeaders.CONTENT_ENCODING))) {
requestBody = gzipBody(requestBody);
}
return requestBody;
}
private static RequestBody gzipBody(final RequestBody body) {
return new RequestBody() {
@Override
public MediaType contentType() {
return body.contentType();
}
@Override
public long contentLength() {
return -1; // We don't know the compressed length in advance!
}
@Override
public void writeTo(@NotNull BufferedSink sink) throws IOException {
BufferedSink gzipSink = Okio.buffer(new GzipSink(sink));
body.writeTo(gzipSink);
gzipSink.close();
}
};
}
private static String parseContentType(RestHeaders headers) {
if (headers != null) {
String contentType = headers.get(RestHeaders.CONTENT_TYPE);
if (contentType != null) {
return contentType;
}
}
return RestHeaders.APPLICATION_JSON;
}
private OkHttpClient buildOkHttpClient(RestClientConfig config) {
OkHttpClient.Builder builder = new OkHttpClient.Builder();
if (config.getTimeout() != null) {
builder.connectTimeout(config.getTimeout(), TimeUnit.MILLISECONDS)
.readTimeout(config.getTimeout(), TimeUnit.MILLISECONDS);
}
if (config.getConnectTimeout() != null) {
builder.connectTimeout(config.getConnectTimeout(), TimeUnit.MILLISECONDS);
}
if (config.getReadTimeout() != null) {
builder.readTimeout(config.getReadTimeout(), TimeUnit.MILLISECONDS);
}
if (config.getMaxIdleConns() != null || config.getIdleTime() != null) {
ConnectionPool connectionPool = new ConnectionPool(config.getMaxIdleConns(),
config.getIdleTime(),
TimeUnit.SECONDS);
builder.connectionPool(connectionPool);
}
// auth header interceptor
if (StringUtils.isNotBlank(config.getUser()) &&
StringUtils.isNotBlank(config.getPassword())) {
builder.addInterceptor(new OkHttpBasicAuthInterceptor(config.getUser(),
config.getPassword()));
}
if (StringUtils.isNotBlank(config.getToken())) {
builder.addInterceptor(new OkHttpTokenInterceptor(config.getToken()));
}
// ssl
configSsl(builder, this.baseUrl, config.getTrustStoreFile(),
config.getTrustStorePassword());
// Execute builder callback before builder.build() for user configs
if (config.getBuilderCallback() != null) {
config.getBuilderCallback().accept(builder);
}
OkHttpClient okHttpClient = builder.build();
if (config.getMaxConns() != null) {
okHttpClient.dispatcher().setMaxRequests(config.getMaxConns());
}
if (config.getMaxConnsPerRoute() != null) {
okHttpClient.dispatcher().setMaxRequestsPerHost(config.getMaxConnsPerRoute());
}
return okHttpClient;
}
@SneakyThrows
private void configSsl(OkHttpClient.Builder builder, String url, String trustStoreFile,
String trustStorePass) {
if (StringUtils.isBlank(trustStoreFile) || StringUtils.isBlank(trustStorePass)) {
return;
}
X509TrustManager trustManager = trustManagerForCertificates(trustStoreFile, trustStorePass);
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, new TrustManager[]{trustManager}, null);
SSLSocketFactory sslSocketFactory = sslContext.getSocketFactory();
builder.sslSocketFactory(sslSocketFactory, trustManager)
.hostnameVerifier(new HostNameVerifier(url));
}
@Override
public RestResult post(String path, Object object) {
return this.post(path, object, null, null);
}
@Override
public RestResult post(String path, Object object, RestHeaders headers) {
return this.post(path, object, headers, null);
}
@Override
public RestResult post(String path, Object object, Map<String, Object> params) {
return this.post(path, object, null, params);
}
private Request.Builder genRequestBuilder(String path, String id, RestHeaders headers,
Map<String, Object> params) {
HttpUrl.Builder urlBuilder = Objects.requireNonNull(HttpUrl.parse(this.baseUrl))
.newBuilder()
.addPathSegments(path);
if (id != null) {
urlBuilder.addPathSegment(id);
}
if (params != null) {
params.forEach((name, value) -> {
if (value == null) {
return;
}
if (value instanceof Collection) {
for (Object i : (Collection<?>) value) {
urlBuilder.addQueryParameter(name, String.valueOf(i));
}
} else {
urlBuilder.addQueryParameter(name, String.valueOf(value));
}
});
}
Request.Builder builder = newRequestBuilder().url(urlBuilder.build());
if (headers != null) {
builder.headers(headers.toOkHttpHeader());
}
this.attachAuthToRequest(builder);
return builder;
}
/**
* In order to provide subclasses with overloading opportunities
*/
protected Request.Builder newRequestBuilder() {
return new Request.Builder();
}
@SneakyThrows
@Override
public RestResult post(String path, Object object, RestHeaders headers,
Map<String, Object> params) {
Request.Builder requestBuilder = genRequestBuilder(path, null, headers, params);
requestBuilder.post(buildRequestBody(object, headers));
try (Response response = request(requestBuilder)) {
checkStatus(response, 200, 201, 202);
return new RestResult(response);
}
}
@Override
public RestResult put(String path, String id, Object object) {
return this.put(path, id, object, ImmutableMap.of());
}
@Override
public RestResult put(String path, String id, Object object, RestHeaders headers) {
return this.put(path, id, object, headers, null);
}
@Override
public RestResult put(String path, String id, Object object, Map<String, Object> params) {
return this.put(path, id, object, null, params);
}
@SneakyThrows
@Override
public RestResult put(String path, String id, Object object,
RestHeaders headers,
Map<String, Object> params) {
Request.Builder requestBuilder = genRequestBuilder(path, id, headers, params);
requestBuilder.put(buildRequestBody(object, headers));
try (Response response = request(requestBuilder)) {
checkStatus(response, 200, 202);
return new RestResult(response);
}
}
@Override
public RestResult get(String path) {
return this.get(path, null, ImmutableMap.of());
}
@Override
public RestResult get(String path, Map<String, Object> params) {
return this.get(path, null, params);
}
@Override
public RestResult get(String path, String id) {
return this.get(path, id, ImmutableMap.of());
}
@SneakyThrows
private RestResult get(String path, String id, Map<String, Object> params) {
Request.Builder requestBuilder = genRequestBuilder(path, id, null, params);
try (Response response = request(requestBuilder)) {
checkStatus(response, 200);
return new RestResult(response);
}
}
@Override
public RestResult delete(String path, Map<String, Object> params) {
return this.delete(path, null, params);
}
@Override
public RestResult delete(String path, String id) {
return this.delete(path, id, ImmutableMap.of());
}
@SneakyThrows
private RestResult delete(String path, String id,
Map<String, Object> params) {
Request.Builder requestBuilder = genRequestBuilder(path, id, null, params);
requestBuilder.delete();
try (Response response = request(requestBuilder)) {
checkStatus(response, 204, 202);
return new RestResult(response);
}
}
protected abstract void checkStatus(Response response, int... statuses);
@SneakyThrows
protected Response request(Request.Builder requestBuilder) {
return this.client.newCall(requestBuilder.build()).execute();
}
@SneakyThrows
@Override
public void close() {
if (this.client != null) {
this.client.dispatcher().executorService().shutdown();
this.client.connectionPool().evictAll();
if (this.client.cache() != null) {
this.client.cache().close();
}
}
}
public static String encode(String raw) {
try {
return URLEncoder.encode(raw, StandardCharsets.UTF_8.toString()).replace("+", "%2B");
} catch (UnsupportedEncodingException e) {
throw new IllegalArgumentException("Failed to encode string: " + raw, e);
}
}
public void resetAuthContext() {
this.authContext.remove();
}
public String getAuthContext() {
return this.authContext.get();
}
public void setAuthContext(String auth) {
this.authContext.set(auth);
}
private void attachAuthToRequest(Request.Builder builder) {
// Add auth header
String auth = this.getAuthContext();
if (StringUtils.isNotEmpty(auth)) {
builder.addHeader(RestHeaders.AUTHORIZATION, auth);
}
}
@SneakyThrows
private X509TrustManager trustManagerForCertificates(String trustStoreFile,
String trustStorePass) {
char[] password = trustStorePass.toCharArray();
// load keyStore
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
try (FileInputStream in = new FileInputStream(trustStoreFile)) {
keyStore.load(in, password);
}
TrustManagerFactory trustManagerFactory =
TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(keyStore);
TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();
if (trustManagers.length != 1 || !(trustManagers[0] instanceof X509TrustManager)) {
throw new IllegalStateException("Unexpected default trust managers:" +
Arrays.toString(trustManagers));
}
return (X509TrustManager) trustManagers[0];
}
public static class HostNameVerifier implements HostnameVerifier {
private final String url;
public HostNameVerifier(String url) {
if (!url.startsWith("http://") && !url.startsWith("https://")) {
url = "http://" + url;
}
url = URI.create(url).getHost();
this.url = url;
}
@Override
public boolean verify(String hostname, SSLSession session) {
if (!this.url.isEmpty() && this.url.endsWith(hostname)) {
return true;
} else {
HostnameVerifier verifier = HttpsURLConnection.getDefaultHostnameVerifier();
return verifier.verify(hostname, session);
}
}
}
}

View File

@ -1,35 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.rest;
public class ClientException extends RuntimeException {
private static final long serialVersionUID = 814572040103754705L;
public ClientException(String message, Throwable cause) {
super(message, cause);
}
public ClientException(String message, Object... args) {
super(String.format(message, args));
}
public ClientException(String message, Throwable cause, Object... args) {
super(String.format(message, args), cause);
}
}

View File

@ -1,47 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.rest;
import java.io.IOException;
import okhttp3.Credentials;
import okhttp3.Interceptor;
import okhttp3.Request;
import okhttp3.Response;
public class OkHttpBasicAuthInterceptor implements Interceptor {
private final String credentials;
public OkHttpBasicAuthInterceptor(String user, String password) {
this.credentials = Credentials.basic(user, password);
}
@Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
if (request.header(RestHeaders.AUTHORIZATION) == null) {
Request authenticatedRequest = request.newBuilder()
.header(RestHeaders.AUTHORIZATION,
this.credentials)
.build();
return chain.proceed(authenticatedRequest);
}
return chain.proceed(request);
}
}

View File

@ -1,50 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.rest;
import static org.apache.hugegraph.rest.RestHeaders.AUTHORIZATION;
import static org.apache.hugegraph.rest.RestHeaders.BEARER_PREFIX;
import java.io.IOException;
import okhttp3.Interceptor;
import okhttp3.Request;
import okhttp3.Response;
public class OkHttpTokenInterceptor implements Interceptor {
private final String token;
public OkHttpTokenInterceptor(String token) {
this.token = token;
}
@Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
if (request.header(AUTHORIZATION) == null) {
Request authenticatedRequest = request.newBuilder()
.header(AUTHORIZATION,
BEARER_PREFIX + this.token)
.build();
return chain.proceed(authenticatedRequest);
}
return chain.proceed(request);
}
}

View File

@ -1,63 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.rest;
import java.util.Map;
public interface RestClient {
/**
* Post method
*/
RestResult post(String path, Object object);
RestResult post(String path, Object object, RestHeaders headers);
RestResult post(String path, Object object, Map<String, Object> params);
RestResult post(String path, Object object, RestHeaders headers, Map<String, Object> params);
/**
* Put method
*/
RestResult put(String path, String id, Object object);
RestResult put(String path, String id, Object object, RestHeaders headers);
RestResult put(String path, String id, Object object, Map<String, Object> params);
RestResult put(String path, String id, Object object, RestHeaders headers,
Map<String, Object> params);
/**
* Get method
*/
RestResult get(String path);
RestResult get(String path, Map<String, Object> params);
RestResult get(String path, String id);
/**
* Delete method
*/
RestResult delete(String path, Map<String, Object> params);
RestResult delete(String path, String id);
void close();
}

View File

@ -1,53 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.rest;
import java.util.function.Consumer;
import lombok.Builder;
import lombok.Getter;
import lombok.Setter;
import okhttp3.OkHttpClient;
@Builder
@Getter
@Setter
@SuppressWarnings("unused")
public class RestClientConfig {
private String user;
private String password;
private String token;
/**
* @deprecated use connectTimeout and readTimeout instead
*/
@Deprecated
private Integer timeout;
/** unit in milliseconds */
private Integer connectTimeout;
/** unit in milliseconds */
private Integer readTimeout;
private Integer maxConns;
private Integer maxConnsPerRoute;
// unit in seconds
private Integer idleTime = 30;
private Integer maxIdleConns = 5;
private String trustStoreFile;
private String trustStorePassword;
private Consumer<OkHttpClient.Builder> builderCallback;
}

View File

@ -1,90 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.rest;
import java.util.Date;
import java.util.Iterator;
import kotlin.Pair;
public class RestHeaders {
public static final String CONTENT_TYPE = "Content-Type";
public static final String CONTENT_ENCODING = "Content-Encoding";
public static final String AUTHORIZATION = "Authorization";
public static final String APPLICATION_JSON = "application/json";
public static final String BEARER_PREFIX = "Bearer ";
private final okhttp3.Headers.Builder headersBuilder;
public RestHeaders() {
this.headersBuilder = new okhttp3.Headers.Builder();
}
public static RestHeaders convertToRestHeaders(okhttp3.Headers headers) {
RestHeaders restHeaders = new RestHeaders();
if (headers != null) {
Iterator<Pair<String, String>> iter = headers.iterator();
while (iter.hasNext()) {
Pair<String, String> pair = iter.next();
restHeaders.add(pair.getFirst(), pair.getSecond());
}
}
return restHeaders;
}
public String get(String key) {
return this.headersBuilder.get(key);
}
public Date getDate(String key) {
return this.headersBuilder.build().getDate(key);
}
public RestHeaders add(String key, String value) {
this.headersBuilder.add(key, value);
return this;
}
public RestHeaders add(String key, Date value) {
this.headersBuilder.add(key, value);
return this;
}
@Override
public int hashCode() {
return this.toOkHttpHeader().hashCode();
}
@Override
public boolean equals(Object obj) {
if (obj instanceof RestHeaders) {
return this.toOkHttpHeader().equals(((RestHeaders) obj).toOkHttpHeader());
}
return false;
}
public okhttp3.Headers toOkHttpHeader() {
return this.headersBuilder.build();
}
}

View File

@ -1,114 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.rest;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.Module;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.SneakyThrows;
import okhttp3.Response;
public class RestResult {
private static final ObjectMapper MAPPER = new ObjectMapper();
private final int status;
private final RestHeaders headers;
private final String content;
public RestResult(Response response) {
this(response.code(), getResponseContent(response),
RestHeaders.convertToRestHeaders(response.headers()));
}
public RestResult(int status, String content, RestHeaders headers) {
this.status = status;
this.headers = headers;
this.content = content;
}
@SneakyThrows
private static String getResponseContent(Response response) {
return response.body().string();
}
public static void registerModule(Module module) {
MAPPER.registerModule(module);
}
public int status() {
return this.status;
}
public RestHeaders headers() {
return this.headers;
}
public String content() {
return this.content;
}
public <T> T readObject(Class<T> clazz) {
try {
return MAPPER.readValue(this.content, clazz);
} catch (Exception e) {
throw new SerializeException("Failed to deserialize: %s", e, this.content);
}
}
@SuppressWarnings("deprecation")
public <T> List<T> readList(String key, Class<T> clazz) {
try {
JsonNode root = MAPPER.readTree(this.content);
JsonNode element = root.get(key);
if (element == null) {
throw new SerializeException("Can't find value of the key: %s in json.", key);
}
JavaType type = MAPPER.getTypeFactory()
.constructParametrizedType(ArrayList.class,
List.class, clazz);
return MAPPER.convertValue(element, type);
} catch (IOException e) {
throw new SerializeException("Failed to deserialize %s", e, this.content);
}
}
@SuppressWarnings("deprecation")
public <T> List<T> readList(Class<T> clazz) {
JavaType type = MAPPER.getTypeFactory()
.constructParametrizedType(ArrayList.class,
List.class, clazz);
try {
return MAPPER.readValue(this.content, type);
} catch (IOException e) {
throw new SerializeException("Failed to deserialize %s", e, this.content);
}
}
@Override
public String toString() {
return String.format("{status=%s, headers=%s, content=%s}", this.status, this.headers,
this.content);
}
}

View File

@ -1,35 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.rest;
public class SerializeException extends ClientException {
private static final long serialVersionUID = -4622753445618619311L;
public SerializeException(String message, Throwable e) {
super(message, e);
}
public SerializeException(String message, Object... args) {
super(message, args);
}
public SerializeException(String message, Throwable e, Object... args) {
super(message, e, args);
}
}

View File

@ -1,164 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.testutil;
import java.util.function.Consumer;
import java.util.function.Function;
import org.hamcrest.BaseMatcher;
import org.hamcrest.CoreMatchers;
import org.hamcrest.Description;
public class Assert extends org.junit.Assert {
@FunctionalInterface
public interface ThrowableRunnable {
void run() throws Throwable;
}
@FunctionalInterface
public interface ThrowableConsumer<T> {
void accept(T t) throws Throwable;
}
public static void assertThrows(Class<? extends Throwable> clazz,
ThrowableRunnable runnable,
Consumer<Throwable> exceptionConsumer) {
Throwable expectedException = assertThrows(clazz, runnable);
assert expectedException != null;
exceptionConsumer.accept(expectedException);
}
public static Throwable assertThrows(Class<? extends Throwable> clazz,
ThrowableRunnable runnable) {
try {
// expect throwing here
runnable.run();
} catch (Throwable e) {
if (!clazz.isInstance(e)) {
// exception type not matched
Assert.fail(String.format("Bad exception type %s(expected %s)",
e.getClass().getName(), clazz.getName()));
}
return e;
}
// no exception
Assert.fail(String.format("No exception was thrown(expected %s)",
clazz.getName()));
// unavailable
assert false;
return null;
}
public static void assertEquals(byte expected, Object actual) {
org.junit.Assert.assertEquals(expected, actual);
}
public static void assertEquals(short expected, Object actual) {
org.junit.Assert.assertEquals(expected, actual);
}
public static void assertEquals(char expected, Object actual) {
org.junit.Assert.assertEquals(expected, actual);
}
public static void assertEquals(int expected, Object actual) {
org.junit.Assert.assertEquals(expected, actual);
}
public static void assertEquals(long expected, Object actual) {
org.junit.Assert.assertEquals(expected, actual);
}
public static void assertEquals(float expected, Object actual) {
org.junit.Assert.assertEquals(expected, actual);
}
public static void assertEquals(double expected, Object actual) {
org.junit.Assert.assertEquals(expected, actual);
}
@SuppressWarnings("deprecation")
public static void assertGt(Number expected, Object actual) {
org.junit.Assert.assertThat(actual, new NumberMatcher(expected, cmp -> {
return cmp > 0;
}, ">"));
}
@SuppressWarnings("deprecation")
public static void assertGte(Number expected, Object actual) {
org.junit.Assert.assertThat(actual, new NumberMatcher(expected, cmp -> {
return cmp >= 0;
}, ">="));
}
@SuppressWarnings("deprecation")
public static void assertLt(Number expected, Object actual) {
org.junit.Assert.assertThat(actual, new NumberMatcher(expected, cmp -> {
return cmp < 0;
}, "<"));
}
@SuppressWarnings("deprecation")
public static void assertLte(Number expected, Object actual) {
org.junit.Assert.assertThat(actual, new NumberMatcher(expected, cmp -> {
return cmp <= 0;
}, "<="));
}
@SuppressWarnings("deprecation")
public static void assertContains(String sub, String actual) {
org.junit.Assert.assertThat(actual, CoreMatchers.containsString(sub));
}
@SuppressWarnings("deprecation")
public static void assertInstanceOf(Class<?> clazz, Object object) {
org.junit.Assert.assertThat(object, CoreMatchers.instanceOf(clazz));
}
private static class NumberMatcher extends BaseMatcher<Object> {
private final String symbol;
private final Number expected;
private final Function<Integer, Boolean> cmp;
NumberMatcher(Number expected, Function<Integer, Boolean> cmp, String symbol) {
this.expected = expected;
this.cmp = cmp;
this.symbol = symbol;
}
@SuppressWarnings("unchecked")
@Override
public boolean matches(Object actual) {
Assert.assertInstanceOf(this.expected.getClass(), actual);
Assert.assertInstanceOf(Comparable.class, actual);
int cmp = ((Comparable<Number>) actual).compareTo(this.expected);
return this.cmp.apply(cmp);
}
@Override
public void describeTo(Description desc) {
desc.appendText("a number ").appendText(this.symbol)
.appendText(" ").appendText(this.expected.toString());
}
}
}

View File

@ -1,184 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.testutil;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.Objects;
import org.apache.hugegraph.util.E;
import com.google.common.primitives.Primitives;
public class Whitebox {
public static final char SEPARATOR = '.';
public static void setInternalState(Object target,
String fieldName, Object value) {
assert target != null;
assert fieldName != null;
int sep = fieldName.lastIndexOf(SEPARATOR);
if (sep > 0) {
String prefix = fieldName.substring(0, sep);
Object result = getInternalState(target, prefix);
E.checkArgument(result != null,
"Can't set value on null field: `%s.%s`",
target, prefix);
target = result;
fieldName = fieldName.substring(sep + 1);
}
Class<?> c = target instanceof Class<?> ?
(Class<?>) target : target.getClass();
try {
Field f = getFieldFromHierarchy(c, fieldName);
f.setAccessible(true);
f.set(target, value);
} catch (Exception e) {
throw new RuntimeException(String.format(
"Can't set value of '%s' against object '%s'",
fieldName, target), e);
}
}
public static <T> T getInternalState(Object target, String fieldName) {
assert fieldName != null;
int sep = fieldName.indexOf(SEPARATOR);
if (sep > 0) {
String field = fieldName.substring(0, sep);
Object value = getInternalState(target, field);
field = fieldName.substring(sep + 1);
return getInternalState(value, field);
}
Class<?> c = target instanceof Class<?> ?
(Class<?>) target : target.getClass();
try {
Field f = getFieldFromHierarchy(c, fieldName);
f.setAccessible(true);
@SuppressWarnings("unchecked")
T result = (T) f.get(target);
return result;
} catch (Exception e) {
throw new RuntimeException(String.format(
"Unable to get internal state on field '%s' of %s",
fieldName, target), e);
}
}
private static Field getFieldFromHierarchy(Class<?> clazz, String field) {
Field f = getField(clazz, field);
while (f == null && clazz != Object.class) {
clazz = clazz.getSuperclass();
f = getField(clazz, field);
}
if (f == null) {
throw new RuntimeException(String.format(
"Not declared field '%s' in class '%s'",
field, clazz.getSimpleName()));
}
return f;
}
private static Field getField(Class<?> clazz, String field) {
try {
return clazz.getDeclaredField(field);
} catch (NoSuchFieldException e) {
return null;
}
}
public static <T> T invokeStatic(Class<?> clazz, String methodName,
Object... args) {
return invoke(clazz, methodName, (Object) null, args);
}
public static <T> T invokeStatic(Class<?> clazz, Class<?>[] classes,
String methodName, Object... args) {
return invoke(clazz, classes, methodName, (Object) null, args);
}
public static <T> T invoke(Object owner, String field,
String methodName, Object... args) {
Object self = getInternalState(owner, field);
Objects.requireNonNull(self);
return invoke(self.getClass(), methodName, self, args);
}
public static <T> T invoke(Object owner, String field, Class<?>[] classes,
String methodName, Object... args) {
Object self = getInternalState(owner, field);
Objects.requireNonNull(self);
return invoke(self.getClass(), classes, methodName, self, args);
}
public static <T> T invoke(Class<?> clazz, String methodName,
Object self, Object... args) {
Class<?>[] classes = new Class<?>[args.length];
int i = 0;
for (Object arg : args) {
E.checkArgument(arg != null, "The argument can't be null");
classes[i++] = Primitives.unwrap(arg.getClass());
}
return invoke(clazz, classes, methodName, self, args);
}
public static <T> T invoke(Class<?> clazz, Class<?>[] classes,
String methodName, Object self, Object... args) {
Method method = method(clazz, methodName, classes);
try {
method.setAccessible(true);
@SuppressWarnings("unchecked")
T result = (T) method.invoke(self, args);
return result;
} catch (IllegalAccessException e) {
throw new RuntimeException(String.format(
"Can't invoke method '%s' of class '%s': %s",
methodName, clazz, e.getMessage()), e);
} catch (InvocationTargetException e) {
Throwable target = e.getTargetException();
if (target instanceof RuntimeException) {
throw (RuntimeException) target;
}
throw new RuntimeException(String.format(
"Can't invoke method '%s' of class '%s': %s",
methodName, clazz, target.getMessage()), target);
}
}
public static Method method(Class<?> clazz, String methodName,
Class<?>[] argsClasses) {
try {
return clazz.getDeclaredMethod(methodName, argsClasses);
} catch (NoSuchMethodException e) {
Class<?> superclass = clazz.getSuperclass();
if (superclass != null) {
try {
return method(superclass, methodName, argsClasses);
} catch (Exception ignored) {
// pass
}
}
throw new RuntimeException(String.format(
"Can't find method '%s' with args %s of class '%s'",
methodName, Arrays.asList(argsClasses), clazz), e);
}
}
}

View File

@ -1,103 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.util.Arrays;
import java.util.Comparator;
import org.apache.commons.codec.DecoderException;
import org.apache.commons.codec.binary.Hex;
import com.google.common.primitives.UnsignedBytes;
/**
* TODO: extends com.google.common.primitives.Bytes
*/
public final class Bytes {
public static final long BASE = 1024L;
public static final long KB = BASE;
public static final long MB = KB * BASE;
public static final long GB = MB * BASE;
public static final long TB = GB * KB;
public static final long PB = GB * MB;
public static final long EB = GB * GB;
private static final Comparator<byte[]> CMP = UnsignedBytes.lexicographicalComparator();
public static int compare(byte[] bytes1, byte[] bytes2) {
return CMP.compare(bytes1, bytes2);
}
public static byte[] concat(byte[] bytes1, byte[] bytes2) {
byte[] result = new byte[bytes1.length + bytes2.length];
System.arraycopy(bytes1, 0, result, 0, bytes1.length);
System.arraycopy(bytes2, 0, result, bytes1.length, bytes2.length);
return result;
}
public static boolean prefixWith(byte[] bytes, byte[] prefix) {
if (bytes.length < prefix.length) {
return false;
}
for (int i = 0; i < prefix.length; i++) {
if (bytes[i] != prefix[i]) {
return false;
}
}
return true;
}
public static boolean contains(byte[] bytes, byte value) {
for (byte b : bytes) {
if (b == value) {
return true;
}
}
return false;
}
public static int indexOf(byte[] bytes, byte value) {
for (int i = 0; i < bytes.length; i++) {
if (bytes[i] == value) {
return i;
}
}
return -1;
}
public static boolean equals(byte[] bytes1, byte[] bytes2) {
return Arrays.equals(bytes1, bytes2);
}
public static String toHex(byte b) {
return toHex(new byte[]{b});
}
public static String toHex(byte[] bytes) {
return new String(Hex.encodeHex(bytes));
}
public static byte[] fromHex(String hex) {
try {
return Hex.decodeHex(hex.toCharArray());
} catch (DecoderException e) {
throw new RuntimeException("Failed to decode hex: " + hex, e);
}
}
}

View File

@ -1,37 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import static java.lang.System.exit;
import java.net.InetAddress;
import java.net.Socket;
public final class CheckSocket {
public static void main(String[] args) {
try {
// Check if the socket connection can be closed normally
new Socket(InetAddress.getByName(args[0]), Integer.parseInt(args[1])).close();
exit(0);
} catch (Exception e) {
e.printStackTrace();
exit(-1);
}
}
}

View File

@ -1,448 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.lang.reflect.Array;
import java.util.ArrayDeque;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.Deque;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ThreadLocalRandom;
import java.util.function.Function;
public final class CollectionUtil {
public static <T> Set<T> toSet(Object object) {
E.checkNotNull(object, "object");
Set<T> set = InsertionOrderUtil.newSet();
fillCollection(set, object);
return set;
}
public static <T> List<T> toList(Object object) {
E.checkNotNull(object, "object");
List<T> list = new ArrayList<>();
fillCollection(list, object);
return list;
}
@SuppressWarnings("unchecked")
private static <T> void fillCollection(Collection<T> collection,
Object object) {
if (object.getClass().isArray()) {
int length = Array.getLength(object);
for (int i = 0; i < length; i++) {
collection.add((T) Array.get(object, i));
}
} else if (object instanceof Collection) {
collection.addAll((Collection<T>) object);
} else {
collection.add((T) object);
}
}
public static <T> boolean prefixOf(List<T> prefix, List<T> all) {
E.checkNotNull(prefix, "prefix");
E.checkNotNull(all, "all");
if (prefix.size() > all.size()) {
return false;
}
for (int i = 0; i < prefix.size(); i++) {
T first = prefix.get(i);
T second = all.get(i);
if (first == second) {
continue;
}
if (first == null || !first.equals(second)) {
return false;
}
}
return true;
}
public static Set<Integer> randomSet(int min, int max, int count) {
E.checkArgument(max > min, "Invalid min/max: %s/%s", min, max);
E.checkArgument(0 < count && count <= max - min,
"Invalid count %s", count);
Set<Integer> randoms = new HashSet<>();
while (randoms.size() < count) {
randoms.add(ThreadLocalRandom.current().nextInt(min, max));
}
return randoms;
}
public static boolean allUnique(Collection<?> collection) {
HashSet<Object> set = new HashSet<>(collection.size());
for (Object elem : collection) {
if (!set.add(elem)) {
return false;
}
}
return true;
}
/**
* Get sub-set of a set.
* @param original original set
* @param from index of start position
* @param to index of end position(exclude), but -1 means the last element
* @param <T> element type of set
* @return sub-set of original set [from, to)
*/
public static <T> Set<T> subSet(Set<T> original, int from, int to) {
E.checkArgument(from >= 0,
"Invalid from parameter of subSet(): %s", from);
if (to < 0) {
to = original.size();
} else {
E.checkArgument(to >= from,
"Invalid to parameter of subSet(): %s", to);
}
List<T> list = new ArrayList<>(original);
return new LinkedHashSet<>(list.subList(from, to));
}
public static <T> Set<T> union(Collection<T> first, Collection<T> second) {
E.checkNotNull(first, "first");
E.checkNotNull(second, "second");
HashSet<T> results = new HashSet<>(first);
results.addAll(second);
return results;
}
/**
* Find the intersection of two collections, the original collections
* will not be modified
* @param first the first collection
* @param second the second collection
* @param <T> element type of collection
* @return intersection of the two collections
*/
public static <T> Collection<T> intersect(Collection<T> first,
Collection<T> second) {
E.checkNotNull(first, "first");
E.checkNotNull(second, "second");
HashSet<T> results;
if (first instanceof HashSet) {
@SuppressWarnings("unchecked")
HashSet<T> clone = (HashSet<T>) ((HashSet<T>) first).clone();
results = clone;
} else {
results = new HashSet<>(first);
}
results.retainAll(second);
return results;
}
/**
* Find the intersection of two collections, note that the first collection
* will be modified and store the results
* @param first the first collection, it will be modified
* @param second the second collection
* @param <T> element type of collection
* @return intersection of the two collections
*/
public static <T> Collection<T> intersectWithModify(Collection<T> first,
Collection<T> second) {
E.checkNotNull(first, "first");
E.checkNotNull(second, "second");
first.retainAll(second);
return first;
}
public static <T> boolean hasIntersection(List<T> first, Set<T> second) {
E.checkNotNull(first, "first");
E.checkNotNull(second, "second");
for (T firstElem : first) {
if (second.contains(firstElem)) {
return true;
}
}
return false;
}
public static <T> boolean hasIntersection(Set<T> first, Set<T> second) {
E.checkNotNull(first, "first");
E.checkNotNull(second, "second");
if (first.size() <= second.size()) {
for (T firstElem : first) {
if (second.contains(firstElem)) {
return true;
}
}
} else {
for (T secondElem : second) {
if (first.contains(secondElem)) {
return true;
}
}
}
return false;
}
public static <K extends Comparable<? super K>, V> Map<K, V> sortByKey(
Map<K, V> map, boolean incr) {
List<Map.Entry<K, V>> list = new ArrayList<>(map.entrySet());
if (incr) {
list.sort(Map.Entry.comparingByKey());
} else {
list.sort(Collections.reverseOrder(Map.Entry.comparingByKey()));
}
Map<K, V> result = new LinkedHashMap<>();
for (Map.Entry<K, V> entry : list) {
result.put(entry.getKey(), entry.getValue());
}
return result;
}
public static <K, V extends Comparable<? super V>> Map<K, V> sortByValue(
Map<K, V> map, boolean incr) {
List<Map.Entry<K, V>> list = new ArrayList<>(map.entrySet());
if (incr) {
list.sort(Map.Entry.comparingByValue());
} else {
list.sort(Collections.reverseOrder(Map.Entry.comparingByValue()));
}
Map<K, V> result = new LinkedHashMap<>();
for (Map.Entry<K, V> entry : list) {
result.put(entry.getKey(), entry.getValue());
}
return result;
}
/**
* Cross combine the A(n, n) combinations of each part in parts
* @param parts a list of part, like [{a,b}, {1,2}, {x,y}]
* @return List<List<Integer>> all combinations like
* [{a,b,1,2,x,y}, {a,b,1,2,y,x}, {a,b,2,1,x,y}, {a,b,2,1,y,x}...]
*/
public static <T> List<List<T>> crossCombineParts(List<List<T>> parts) {
List<List<T>> results = new ArrayList<>();
Deque<List<T>> selected = new ArrayDeque<>();
crossCombineParts(parts, 0, selected, results);
return results;
}
private static <T> void crossCombineParts(List<List<T>> parts,
int level,
Deque<List<T>> selected,
List<List<T>> results) {
assert level < parts.size();
List<T> part = parts.get(level);
for (List<T> combination : anm(part)) {
selected.addLast(combination);
if (level < parts.size() - 1) {
crossCombineParts(parts, level + 1, selected, results);
} else if (level == parts.size() - 1) {
results.add(flatToList(selected));
}
selected.removeLast();
}
}
private static <T> List<T> flatToList(Deque<List<T>> parts) {
List<T> list = new ArrayList<>();
for (List<T> part : parts) {
list.addAll(part);
}
return list;
}
/**
* Traverse C(n, m) combinations of a list
* @param all list to contain all items for combination
* @param n m of C(n, m)
* @param m n of C(n, m)
* @return List<List<T>> all combinations
*/
public static <T> List<List<T>> cnm(List<T> all, int n, int m) {
List<List<T>> combs = new ArrayList<>();
cnm(all, n, m, comb -> {
combs.add(comb);
return false;
});
return combs;
}
/**
* Traverse C(n, m) combinations of a list to find first matched
* result combination and call back with the result.
* @param all list to contain all items for combination
* @param n m of C(n, m)
* @param m n of C(n, m)
* @return true if matched any kind of items combination else false, the
* callback can always return false if you want to traverse all combinations
*/
public static <T> boolean cnm(List<T> all, int n, int m,
Function<List<T>, Boolean> callback) {
return cnm(all, n, m, 0, null, callback);
}
/**
* Traverse C(n, m) combinations of a list to find first matched
* result combination and call back with the result.
* @param all list to contain all items for combination
* @param n n of C(n, m)
* @param m m of C(n, m)
* @param current current position in list
* @param selected list to contain selected items
* @return true if matched any kind of items combination else false, the
* callback can always return false if you want to traverse all combinations
*/
private static <T> boolean cnm(List<T> all, int n, int m,
int current, List<T> selected,
Function<List<T>, Boolean> callback) {
assert n <= all.size();
assert m <= n;
assert current <= all.size();
if (selected == null) {
selected = new ArrayList<>(m);
}
if (m == 0) {
assert selected.size() > 0 : selected;
// All n items are selected
List<T> tmpResult = Collections.unmodifiableList(selected);
return callback.apply(tmpResult);
}
if (n == m) {
// No choice, select all n items, we don't update the `result` here
List<T> tmpResult = new ArrayList<>(selected);
tmpResult.addAll(all.subList(current, all.size()));
return callback.apply(tmpResult);
}
if (current >= all.size()) {
// Reach the end of items
return false;
}
// Select current item, continue to select C(m-1, n-1)
int index = selected.size();
selected.add(all.get(current));
++current;
if (cnm(all, n - 1, m - 1, current, selected, callback)) {
// NOTE: we can pop the tailing items if want to keep result clear
return true;
}
// Not select current item, pop it and continue to select C(m-1, n)
selected.remove(index);
assert selected.size() == index : selected;
return cnm(all, n - 1, m, current, selected, callback);
}
/**
* Traverse A(n, m) combinations of a list with n = m = all.size()
* @param all list to contain all items for combination
* @return List<List<T>> all combinations
*/
public static <T> List<List<T>> anm(List<T> all) {
return anm(all, all.size(), all.size());
}
/**
* Traverse A(n, m) combinations of a list
* @param all list to contain all items for combination
* @param n m of A(n, m)
* @param m n of A(n, m)
* @return List<List<T>> all combinations
*/
public static <T> List<List<T>> anm(List<T> all, int n, int m) {
List<List<T>> combs = new ArrayList<>();
anm(all, n, m, comb -> {
combs.add(comb);
return false;
});
return combs;
}
/**
* Traverse A(n, m) combinations of a list to find first matched
* result combination and call back with the result.
* @param all list to contain all items for combination
* @param n m of A(n, m)
* @param m n of A(n, m)
* @return true if matched any kind of items combination else false, the
* callback can always return false if you want to traverse all combinations
*/
public static <T> boolean anm(List<T> all, int n, int m,
Function<List<T>, Boolean> callback) {
return anm(all, n, m, null, callback);
}
/**
* Traverse A(n, m) combinations of a list to find first matched
* result combination and call back with the result.
* @param all list to contain all items for combination
* @param n m of A(n, m)
* @param m n of A(n, m)
* @param selected list to contain selected items
* @return true if matched any kind of items combination else false, the
* callback can always return false if you want to traverse all combinations
*/
private static <T> boolean anm(List<T> all, int n, int m,
List<Integer> selected,
Function<List<T>, Boolean> callback) {
assert n <= all.size();
assert m <= n;
if (selected == null) {
selected = new ArrayList<>(m);
}
if (m == 0) {
// All n items are selected
List<T> tmpResult = new ArrayList<>();
for (int i : selected) {
tmpResult.add(all.get(i));
}
return callback.apply(tmpResult);
}
for (int i = 0; i < all.size(); i++) {
if (selected.contains(i)) {
continue;
}
int index = selected.size();
selected.add(i);
// Select current item, continue to select A(m-1, n-1)
if (anm(all, n - 1, m - 1, selected, callback)) {
return true;
}
selected.remove(index);
assert selected.size() == index : selected;
}
return false;
}
}

View File

@ -1,77 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.util.Date;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import org.apache.hugegraph.date.SafeDateFormat;
import com.google.common.collect.ImmutableMap;
public final class DateUtil {
public static final Date DATE_ZERO = new Date(0L);
private static final Map<String, String> VALID_DFS = ImmutableMap.of(
"^\\d{4}-\\d{1,2}-\\d{1,2}",
"yyyy-MM-dd",
"^\\d{4}-\\d{1,2}-\\d{1,2}\\s\\d{2}:\\d{2}:\\d{2}",
"yyyy-MM-dd HH:mm:ss",
"^\\d{4}-\\d{1,2}-\\d{1,2}\\s\\d{2}:\\d{2}:\\d{2}\\.\\d{1,3}",
"yyyy-MM-dd HH:mm:ss.SSS"
);
private static final Map<String, SafeDateFormat> DATE_FORMATS = new ConcurrentHashMap<>();
public static Date parse(String value) {
for (Map.Entry<String, String> entry : VALID_DFS.entrySet()) {
if (value.matches(entry.getKey())) {
return parse(value, entry.getValue());
}
}
throw new IllegalArgumentException(String.format(
"Expected date format is: %s, but got '%s'", VALID_DFS.values(), value));
}
public static Date parse(String value, String df) {
SafeDateFormat dateFormat = getDateFormat(df);
return dateFormat.parse(value);
}
public static Date now() {
return new Date();
}
private static SafeDateFormat getDateFormat(String df) {
SafeDateFormat dateFormat = DATE_FORMATS.get(df);
if (dateFormat == null) {
dateFormat = new SafeDateFormat(df);
SafeDateFormat previous = DATE_FORMATS.putIfAbsent(df, dateFormat);
if (previous != null) {
dateFormat = previous;
}
}
return dateFormat;
}
public static Object toPattern(String df) {
SafeDateFormat dateFormat = getDateFormat(df);
return dateFormat.toPattern();
}
}

View File

@ -1,66 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.util.Collection;
import javax.annotation.Nullable;
import com.google.common.base.Preconditions;
public final class E {
public static void checkNotNull(Object object, String elem) {
Preconditions.checkNotNull(object, "The '%s' can't be null", elem);
}
public static void checkNotNull(Object object, String elem, String owner) {
Preconditions.checkNotNull(object,
"The '%s' of '%s' can't be null",
elem, owner);
}
public static void checkNotEmpty(Collection<?> collection, String elem) {
Preconditions.checkArgument(!collection.isEmpty(),
"The '%s' can't be empty", elem);
}
public static void checkNotEmpty(Collection<?> collection, String elem, String owner) {
Preconditions.checkArgument(!collection.isEmpty(),
"The '%s' of '%s' can't be empty",
elem, owner);
}
public static void checkArgument(boolean expression,
@Nullable String message,
@Nullable Object... args) {
Preconditions.checkArgument(expression, message, args);
}
public static void checkArgumentNotNull(Object object,
@Nullable String message,
@Nullable Object... args) {
Preconditions.checkArgument(object != null, message, args);
}
public static void checkState(boolean expression,
@Nullable String message,
@Nullable Object... args) {
Preconditions.checkState(expression, message, args);
}
}

View File

@ -1,49 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
public final class ExceptionUtil {
public static Throwable rootCause(Throwable e) {
Throwable cause = e;
while (cause.getCause() != null) {
cause = cause.getCause();
}
return cause;
}
public static RuntimeException transToRuntimeException(Throwable e) {
if (e instanceof RuntimeException) {
return (RuntimeException) e;
}
return new RuntimeException(rootCause(e).getMessage(), e);
}
public static <T> T futureGet(Future<T> future) {
try {
return future.get();
} catch (InterruptedException e) {
throw ExceptionUtil.transToRuntimeException(e);
} catch (ExecutionException e) {
throw ExceptionUtil.transToRuntimeException(e.getCause());
}
}
}

View File

@ -1,111 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.LinkedBlockingQueue;
import org.apache.commons.lang3.concurrent.BasicThreadFactory;
import org.apache.hugegraph.concurrent.PausableScheduledThreadPool;
public final class ExecutorUtil {
public static ThreadPoolExecutor newDynamicThreadExecutor(String name,
int corePoolSize,
int maximumPoolSize) {
long keepAliveTime = 60L;
TimeUnit unit = TimeUnit.SECONDS;
ThreadFactory factory = new BasicThreadFactory.Builder()
.namingPattern(name)
.build();
CustomBlockingQueue<Runnable> workQueue = new CustomBlockingQueue<>();
ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(
corePoolSize,
maximumPoolSize,
keepAliveTime,
unit,
workQueue,
factory,
new ThreadPoolExecutor.CallerRunsPolicy()
);
workQueue.setThreadPoolExecutor(threadPoolExecutor);
return threadPoolExecutor;
}
static class CustomBlockingQueue<E> extends LinkedBlockingQueue<E> {
private ThreadPoolExecutor threadPoolExecutor;
public void setThreadPoolExecutor(ThreadPoolExecutor threadPoolExecutor) {
this.threadPoolExecutor = threadPoolExecutor;
}
@Override
public boolean offer(E e) {
if (threadPoolExecutor.getPoolSize() < threadPoolExecutor.getMaximumPoolSize()) {
return false;
}
return super.offer(e);
}
}
public static ExecutorService newFixedThreadPool(String name) {
return newFixedThreadPool(1, name);
}
public static ExecutorService newFixedThreadPool(int size, String name) {
ThreadFactory factory = new BasicThreadFactory.Builder()
.namingPattern(name)
.build();
return Executors.newFixedThreadPool(size, factory);
}
public static ScheduledExecutorService newScheduledThreadPool(String name) {
return newScheduledThreadPool(1, name);
}
public static ScheduledExecutorService newScheduledThreadPool(int size,
String name) {
ThreadFactory factory = new BasicThreadFactory.Builder()
.namingPattern(name)
.build();
return Executors.newScheduledThreadPool(size, factory);
}
public static PausableScheduledThreadPool newPausableScheduledThreadPool(
String name) {
return newPausableScheduledThreadPool(1, name);
}
public static PausableScheduledThreadPool newPausableScheduledThreadPool(
int size, String name) {
ThreadFactory factory = new BasicThreadFactory.Builder()
.namingPattern(name)
.build();
return new PausableScheduledThreadPool(size, factory);
}
}

View File

@ -1,44 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.nio.charset.Charset;
import com.google.common.base.Charsets;
import com.google.common.hash.Hashing;
public final class HashUtil {
private static final Charset CHARSET = Charsets.UTF_8;
public static byte[] hash(byte[] bytes) {
return Hashing.murmur3_32().hashBytes(bytes).asBytes();
}
public static String hash(String value) {
return Hashing.murmur3_32().hashString(value, CHARSET).toString();
}
public static byte[] hash128(byte[] bytes) {
return Hashing.murmur3_128().hashBytes(bytes).asBytes();
}
public static String hash128(String value) {
return Hashing.murmur3_128().hashString(value, CHARSET).toString();
}
}

View File

@ -1,64 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
public final class InsertionOrderUtil {
public static <K, V> Map<K, V> newMap() {
return new LinkedHashMap<>();
}
public static <K, V> Map<K, V> newMap(int initialCapacity) {
return new LinkedHashMap<>(initialCapacity);
}
public static <K, V> Map<K, V> newMap(Map<K, V> origin) {
return new LinkedHashMap<>(origin);
}
public static <V> Set<V> newSet() {
return new LinkedHashSet<>();
}
public static <V> Set<V> newSet(int initialCapacity) {
return new LinkedHashSet<>(initialCapacity);
}
public static <V> Set<V> newSet(Set<V> origin) {
return new LinkedHashSet<>(origin);
}
public static <V> List<V> newList() {
return new ArrayList<>();
}
public static <V> List<V> newList(int initialCapacity) {
return new ArrayList<>(initialCapacity);
}
public static <V> List<V> newList(List<V> origin) {
return new ArrayList<>(origin);
}
}

View File

@ -1,94 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.io.IOException;
import org.apache.hugegraph.rest.SerializeException;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.Module;
import com.fasterxml.jackson.databind.ObjectMapper;
/**
* Utility class for JSON operations.
*/
public final class JsonUtilCommon {
/**
* ObjectMapper instance used for JSON operations.
*/
private static final ObjectMapper MAPPER = new ObjectMapper();
/**
* Registers a module with the ObjectMapper.
*
* @param module the module to register
*/
public static void registerModule(Module module) {
MAPPER.registerModule(module);
}
/**
* Converts an object to a JSON string.
*
* @param object the object to convert
* @return the JSON string representation of the object
* @throws SerializeException if the object cannot be serialized
*/
public static String toJson(Object object) {
try {
return MAPPER.writeValueAsString(object);
} catch (JsonProcessingException e) {
throw new SerializeException("Failed to serialize object '%s'", e, object);
}
}
/**
* Converts a JSON string to an object of the specified class.
*
* @param json the JSON string
* @param clazz the class of the object
* @return the object represented by the JSON string
* @throws SerializeException if the JSON string cannot be deserialized
*/
public static <T> T fromJson(String json, Class<T> clazz) {
try {
return MAPPER.readValue(json, clazz);
} catch (IOException e) {
throw new SerializeException("Failed to deserialize json '%s'", e, json);
}
}
/**
* Converts a JsonNode to an object of the specified class.
*
* @param node the JsonNode
* @param clazz the class of the object
* @return the object represented by the JsonNode
* @throws SerializeException if the JsonNode cannot be deserialized
*/
public static <T> T convertValue(JsonNode node, Class<T> clazz) {
try {
return MAPPER.convertValue(node, clazz);
} catch (IllegalArgumentException e) {
throw new SerializeException("Failed to deserialize json node '%s'", e, node);
}
}
}

View File

@ -1,32 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public final class Log {
public static Logger logger(String name) {
return LoggerFactory.getLogger(name);
}
public static Logger logger(Class<?> clazz) {
return LoggerFactory.getLogger(clazz);
}
}

View File

@ -1,151 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
public final class LongEncoding {
private static final String B64_SYMBOLS =
"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz~";
private static final String LENGTH_SYMBOLS = "0123456789ABCDEF";
private static final char SORTABLE_NEG = LENGTH_SYMBOLS.charAt(0);
private static final char SIGNED_NEG = '-';
private static final long FULL_LONG = Long.MIN_VALUE;
public static String encodeNumber(Object number) {
Number num = NumericUtil.convertToNumber(number);
long value = NumericUtil.numberToSortableLong(num);
return encodeSortable(value);
}
public static Number decodeNumber(String str, Class<?> clazz) {
long value = decodeSortable(str);
return NumericUtil.sortableLongToNumber(value, clazz);
}
public static String encodeSortable(long num) {
boolean negative = false;
if (num < 0L) {
negative = true;
num += FULL_LONG;
}
String encoded = encode(num, B64_SYMBOLS);
int length = encoded.length();
E.checkArgument(length <= LENGTH_SYMBOLS.length(),
"Length symbols can't represent encoded number '%s'",
encoded);
StringBuilder sb = new StringBuilder(length + 2);
if (negative) {
sb.append(SORTABLE_NEG);
}
char len = LENGTH_SYMBOLS.charAt(length);
sb.append(len);
sb.append(encoded);
return sb.toString();
}
public static long decodeSortable(String str) {
E.checkArgument(str.length() >= 2,
"Length of sortable encoded string must be >=2");
boolean negative = str.charAt(0) == SORTABLE_NEG;
int lengthPos = 0;
if (negative) {
lengthPos = 1;
}
int length = B64_SYMBOLS.indexOf(str.charAt(lengthPos));
E.checkArgument(length == str.length() - lengthPos - 1,
"Can't decode illegal string '%s' with wrong length",
str);
String encoded = str.substring(lengthPos + 1);
long value = decode(encoded, B64_SYMBOLS);
if (negative) {
value -= FULL_LONG;
}
return value;
}
public static String encodeSignedB64(long value) {
boolean negative = false;
if (value < 0L) {
negative = true;
if (value == FULL_LONG) {
return "-80000000000";
}
value = -value;
}
assert value >= 0L : value;
String encoded = encodeB64(value);
return negative ? SIGNED_NEG + encoded : encoded;
}
public static long decodeSignedB64(String value) {
boolean negative = false;
if (!value.isEmpty() && value.charAt(0) == SIGNED_NEG) {
negative = true;
value = value.substring(1);
}
long decoded = decodeB64(value);
return negative ? -decoded : decoded;
}
public static boolean validB64Char(char c) {
return B64_SYMBOLS.indexOf(c) != -1;
}
public static String encodeB64(long num) {
return encode(num, B64_SYMBOLS);
}
public static long decodeB64(String str) {
return decode(str, B64_SYMBOLS);
}
public static long decode(String encoded, String symbols) {
final int B = symbols.length();
E.checkArgument(B > 0, "The symbols parameter can't be empty");
long num = 0L;
for (char ch : encoded.toCharArray()) {
num *= B;
int pos = symbols.indexOf(ch);
if (pos < 0) {
throw new NumberFormatException(String.format(
"Can't decode symbol '%s' in string '%s'",
ch, encoded));
}
num += pos;
}
return num;
}
public static String encode(long num, String symbols) {
final int B = symbols.length();
E.checkArgument(num >= 0L, "Expected non-negative number: %s", num);
E.checkArgument(B > 0, "The symbols parameter can't be empty");
StringBuilder sb = new StringBuilder();
do {
sb.append(symbols.charAt((int) (num % B)));
num /= B;
} while (num != 0L);
return sb.reverse().toString();
}
}

View File

@ -1,372 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.math.BigDecimal;
import java.nio.ByteBuffer;
import java.util.Date;
import java.util.function.Function;
/**
* This file is copied verbatim from Apache Lucene NumericUtils.java
* Only the double/float to sortable long/int conversions are retained.
*/
public final class NumericUtil {
private static final long FULL_LONG = Long.MIN_VALUE;
private static final int FULL_INT = Integer.MIN_VALUE;
private static final byte FULL_BYTE = Byte.MIN_VALUE;
private NumericUtil() {
}
/**
* Converts a <code>double</code> value to a sortable signed
* <code>long</code>. The value is converted by getting their IEEE 754
* floating-point &quot;double format&quot; bit layout and then some bits
* are swapped, to be able to compare the result as long. By this the
* precision is not reduced, but the value can be easily used as a long. The
* sort order (including {@link Double#NaN}) is defined by
* {@link Double#compareTo}; {@code NaN} is greater than positive infinity.
* @param value input double value
* @return output sortable long value
* @see #sortableLongToDouble
*/
public static long doubleToSortableLong(double value) {
return sortableDoubleBits(Double.doubleToLongBits(value));
}
/**
* Converts a sortable <code>long</code> back to a <code>double</code>.
* @param value input double value
* @return output sortable long value
* @see #doubleToSortableLong
*/
public static double sortableLongToDouble(long value) {
return Double.longBitsToDouble(sortableDoubleBits(value));
}
/**
* Converts a <code>float</code> value to a sortable signed
* <code>int</code>. The value is converted by getting their IEEE 754
* floating-point &quot;float format&quot; bit layout and then some bits are
* swapped, to be able to compare the result as int. By this the precision
* is not reduced, but the value can be easily used as an int. The sort order
* (including {@link Float#NaN}) is defined by {@link Float#compareTo};
* {@code NaN} is greater than positive infinity.
* @param value input float value
* @return output sortable int value
* @see #sortableIntToFloat
*/
public static int floatToSortableInt(float value) {
return sortableFloatBits(Float.floatToIntBits(value));
}
/**
* Converts a sortable <code>int</code> back to a <code>float</code>.
* @param value input int value
* @return output sortable float value
* @see #floatToSortableInt
*/
public static float sortableIntToFloat(int value) {
return Float.intBitsToFloat(sortableFloatBits(value));
}
/**
* Converts IEEE 754 representation of a double to sortable order (or back
* to the original)
* @param bits The long format of a double value
* @return The sortable long value
*/
public static long sortableDoubleBits(long bits) {
return bits ^ ((bits >> 63) & 0x7fffffffffffffffL);
}
/**
* Converts IEEE 754 representation of a float to sortable order (or back to
* the original)
* @param bits The int format of a float value
* @return The sortable int value
*/
public static int sortableFloatBits(int bits) {
/*
* Convert to its inverse digits if negative else keep the origin
* NOTE: (bits >> 31) is 0x00000000 if bits >= 0
* (bits >> 31) is 0xFFFFFFFF if bits < 0
*/
return bits ^ ((bits >> 31) & 0x7fffffff);
}
public static long numberToSortableLong(Number number) {
if (number instanceof Double) {
return doubleToSortableLong(number.doubleValue());
} else if (number instanceof Float) {
return floatToSortableInt(number.floatValue());
} else if (number instanceof Long || number instanceof Integer ||
number instanceof Short || number instanceof Byte) {
return number.longValue();
} else if (number instanceof BigDecimal) {
BigDecimal bd = (BigDecimal) number;
boolean intNumber = bd.stripTrailingZeros().scale() <= 0;
return intNumber ? bd.longValueExact() :
doubleToSortableLong(bd.doubleValue());
}
// TODO: support other number types
throw unsupportedNumberType(number);
}
public static Number sortableLongToNumber(long value, Class<?> clazz) {
assert NumericUtil.isNumber(clazz);
if (clazz == Double.class) {
return sortableLongToDouble(value);
} else if (clazz == Float.class) {
return sortableIntToFloat((int) value);
} else if (clazz == Long.class) {
return value;
} else if (clazz == Integer.class) {
return (int) value;
} else if (clazz == Short.class) {
return (short) value;
} else if (clazz == Byte.class) {
return (byte) value;
}
// TODO: support other number types
throw unsupportedNumberType(clazz);
}
public static byte[] numberToSortableBytes(Number number) {
if (number instanceof Long) {
return longToSortableBytes(number.longValue());
} else if (number instanceof Double) {
long value = doubleToSortableLong(number.doubleValue());
return longToSortableBytes(value);
} else if (number instanceof Float) {
int value = floatToSortableInt(number.floatValue());
return intToSortableBytes(value);
} else if (number instanceof Integer || number instanceof Short) {
return intToSortableBytes(number.intValue());
} else if (number instanceof Byte) {
return byteToSortableBytes(number.byteValue());
}
// TODO: support other number types
throw unsupportedNumberType(number);
}
public static Number sortableBytesToNumber(byte[] bytes, Class<?> clazz) {
assert NumericUtil.isNumber(clazz);
if (clazz == Long.class) {
return sortableBytesToLong(bytes);
} else if (clazz == Double.class) {
return sortableLongToDouble(sortableBytesToLong(bytes));
} else if (clazz == Float.class) {
return sortableIntToFloat(sortableBytesToInt(bytes));
} else if (clazz == Integer.class) {
return sortableBytesToInt(bytes);
} else if (clazz == Short.class) {
return (short) sortableBytesToInt(bytes);
} else if (clazz == Byte.class) {
return sortableBytesToByte(bytes);
}
// TODO: support other number types
throw unsupportedNumberType(clazz);
}
public static Number minValueOf(Class<?> clazz) {
E.checkArgumentNotNull(clazz, "The clazz can't be null");
if (Long.class.isAssignableFrom(clazz) ||
Double.class.isAssignableFrom(clazz)) {
return Long.MIN_VALUE;
}
if (Integer.class.isAssignableFrom(clazz) ||
Short.class.isAssignableFrom(clazz) ||
Float.class.isAssignableFrom(clazz)) {
return Integer.MIN_VALUE;
}
if (Byte.class.isAssignableFrom(clazz)) {
return Byte.MIN_VALUE;
}
// TODO: support other number types
throw unsupportedNumberType(clazz);
}
public static Number maxValueOf(Class<?> clazz) {
E.checkArgumentNotNull(clazz, "The clazz can't be null");
if (Long.class.isAssignableFrom(clazz) ||
Double.class.isAssignableFrom(clazz)) {
return Long.MAX_VALUE;
}
if (Integer.class.isAssignableFrom(clazz) ||
Float.class.isAssignableFrom(clazz) ||
Short.class.isAssignableFrom(clazz)) {
return Integer.MAX_VALUE;
}
if (Byte.class.isAssignableFrom(clazz)) {
return Byte.MAX_VALUE;
}
// TODO: support other number types
throw unsupportedNumberType(clazz);
}
public static byte[] longToSortableBytes(long value) {
return longToBytes(value + FULL_LONG);
}
public static long sortableBytesToLong(byte[] bytes) {
return bytesToLong(bytes) - FULL_LONG;
}
public static byte[] intToSortableBytes(int value) {
return intToBytes(value + FULL_INT);
}
public static int sortableBytesToInt(byte[] bytes) {
return bytesToInt(bytes) - FULL_INT;
}
public static byte[] byteToSortableBytes(byte value) {
value += FULL_BYTE;
return new byte[]{value};
}
public static byte sortableBytesToByte(byte[] bytes) {
assert bytes.length == 1;
byte value = bytes[0];
value -= FULL_BYTE;
return value;
}
public static byte[] longToBytes(long value) {
ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
buffer.putLong(value);
return buffer.array();
}
public static long bytesToLong(byte[] bytes) {
assert bytes.length == Long.BYTES;
return ByteBuffer.wrap(bytes).getLong();
}
public static byte[] intToBytes(int value) {
ByteBuffer buffer = ByteBuffer.allocate(Integer.BYTES);
buffer.putInt(value);
return buffer.array();
}
public static int bytesToInt(byte[] bytes) {
assert bytes.length == Integer.BYTES;
return ByteBuffer.wrap(bytes).getInt();
}
public static boolean isNumber(Object value) {
if (value == null) {
return false;
}
return isNumber(value.getClass());
}
public static boolean isNumber(Class<?> clazz) {
if (clazz.isPrimitive()) {
return clazz == int.class || clazz == long.class ||
clazz == float.class || clazz == double.class ||
clazz == short.class || clazz == byte.class;
}
return Number.class.isAssignableFrom(clazz);
}
public static Number convertToNumber(Object value) {
if (value == null) {
return null;
}
Number number;
if (isNumber(value)) {
number = (Number) value;
} else {
if (value instanceof Date) {
number = ((Date) value).getTime();
} else {
// TODO: add some more types to convert
number = new BigDecimal(value.toString());
}
}
return number;
}
/**
* Compare object with a number, the object should be a number,
* or it can be converted to a BigDecimal
* @param first might be number or string
* @param second must be number
* @return 0 if first is numerically equal to second;
* a negative int if first is numerically less than second;
* a positive int if first is numerically greater than second.
*/
public static int compareNumber(Object first, Number second) {
if (first == null) {
E.checkArgument(first != null,
"The first parameter can't be null");
}
if (second == null) {
E.checkArgument(second != null,
"The second parameter can't be null");
}
if (first instanceof Number && first instanceof Comparable &&
first.getClass().equals(second.getClass())) {
@SuppressWarnings("unchecked")
Comparable<Number> cmpFirst = (Comparable<Number>) first;
return cmpFirst.compareTo(second);
}
Function<Object, BigDecimal> toBig = (number) -> {
try {
return new BigDecimal(number.toString());
} catch (NumberFormatException e) {
throw new IllegalArgumentException(String.format(
"Can't compare between '%s' and '%s', " +
"they must be numbers", first, second));
}
};
BigDecimal n1 = toBig.apply(first);
BigDecimal n2 = toBig.apply(second);
return n1.compareTo(n2);
}
private static IllegalArgumentException unsupportedNumberType(Class<?> c) {
return new IllegalArgumentException(String.format(
"Unsupported number type: %s", c.getSimpleName()));
}
private static IllegalArgumentException unsupportedNumberType(Number num) {
return new IllegalArgumentException(String.format(
"Unsupported number type: %s(%s)",
num.getClass().getSimpleName(), num));
}
}

View File

@ -1,103 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.util.HashMap;
import java.util.Map;
import java.util.TreeMap;
import com.google.common.base.Functions;
import com.google.common.collect.Ordering;
public class OrderLimitMap<K extends Comparable<K>, V extends Comparable<V>> extends TreeMap<K, V> {
private static final long serialVersionUID = 756490437953358633L;
private final int capacity;
private final Map<K, V> valueMap;
private static <V extends Comparable<V>> Ordering<? super V> incr() {
return Ordering.from(Comparable::compareTo);
}
private static <V extends Comparable<V>> Ordering<? super V> decr() {
return Ordering.from((V o1, V o2) -> -o1.compareTo(o2));
}
public OrderLimitMap(int capacity) {
this(capacity, false);
}
public OrderLimitMap(int capacity, boolean incr) {
this(capacity, incr ? incr() : decr(), new HashMap<>());
}
private OrderLimitMap(int capacity, Ordering<? super V> ordering,
HashMap<K, V> valueMap) {
/*
* onResultOf: for getting the value for the key from value map
* compound: keep insertion order
*/
super(ordering.onResultOf(Functions.forMap(valueMap))
.compound(Ordering.natural()));
E.checkArgument(capacity > 0, "The capacity must be > 0");
this.capacity = capacity;
this.valueMap = valueMap;
}
@Override
public V put(K k, V v) {
if (this.valueMap.containsKey(k)) {
super.remove(k);
} else if (this.valueMap.size() >= this.capacity) {
K key = super.lastKey();
super.remove(key);
this.valueMap.remove(key);
}
this.valueMap.put(k, v);
return super.put(k, v);
}
@Override
public V get(Object key) {
return this.valueMap.get(key);
}
@Override
public V getOrDefault(Object key, V defaultValue) {
return this.valueMap.getOrDefault(key, defaultValue);
}
@Override
public boolean containsKey(Object key) {
return this.valueMap.containsKey(key);
}
public Map<K, V> topN(int n) {
E.checkArgument(n > 0, "'N' Must be positive, but got '%s'", n);
Map<K, V> top = InsertionOrderUtil.newMap();
int i = 0;
for (Map.Entry<K, V> entry : this.entrySet()) {
top.put(entry.getKey(), entry.getValue());
if (++i >= n) {
break;
}
}
return top;
}
}

View File

@ -1,124 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.io.IOException;
import java.lang.annotation.Annotation;
import java.lang.reflect.Method;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import org.apache.hugegraph.iterator.ExtendableIterator;
import com.google.common.collect.Lists;
import com.google.common.reflect.ClassPath;
import com.google.common.reflect.ClassPath.ClassInfo;
import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtMethod;
import javassist.NotFoundException;
public final class ReflectionUtil {
public static boolean isSimpleType(Class<?> type) {
return type.isPrimitive() ||
type.equals(String.class) ||
type.equals(Boolean.class) ||
type.equals(Character.class) ||
NumericUtil.isNumber(type);
}
public static List<Method> getMethodsAnnotatedWith(
Class<?> type,
Class<? extends Annotation> annotation,
boolean withSuperClass) {
final List<Method> methods = new LinkedList<>();
Class<?> klass = type;
do {
for (Method method : klass.getDeclaredMethods()) {
if (method.isAnnotationPresent(annotation)) {
methods.add(method);
}
}
klass = klass.getSuperclass();
} while (klass != Object.class && withSuperClass);
return methods;
}
public static List<CtMethod> getMethodsAnnotatedWith(
CtClass type,
Class<? extends Annotation> annotation,
boolean withSuperClass)
throws NotFoundException {
final List<CtMethod> methods = new LinkedList<>();
CtClass klass = type;
do {
for (CtMethod method : klass.getDeclaredMethods()) {
if (method.hasAnnotation(annotation)) {
methods.add(method);
}
}
klass = klass.getSuperclass();
} while (klass != null && withSuperClass);
return methods;
}
public static Iterator<ClassInfo> classes(String... packages)
throws IOException {
ClassPath path = ClassPath.from(ReflectionUtil.class.getClassLoader());
ExtendableIterator<ClassInfo> results = new ExtendableIterator<>();
for (String p : packages) {
results.extend(path.getTopLevelClassesRecursive(p).iterator());
}
return results;
}
public static List<String> superClasses(String clazz)
throws NotFoundException {
CtClass klass = ClassPool.getDefault().get(clazz);
CtClass parent = klass.getSuperclass();
List<String> results = new LinkedList<>();
while (parent != null) {
results.add(parent.getName());
parent = parent.getSuperclass();
}
return Lists.reverse(results);
}
public static List<String> nestedClasses(String clazz)
throws NotFoundException {
CtClass klass = ClassPool.getDefault().get(clazz);
List<String> results = new LinkedList<>();
for (CtClass nested : klass.getNestedClasses()) {
results.add(nested.getName());
}
return results;
}
public static String packageName(String clazz) {
int offset = clazz.lastIndexOf(".");
if (offset > 0) {
return clazz.substring(0, offset);
}
return "";
}
}

View File

@ -1,123 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
public final class StringUtil {
public static Chars[] splitToCharsArray(String text, String delimiter) {
E.checkArgument(delimiter.length() > 0,
"The delimiter can't be empty");
Chars[] buffer = new Chars[text.length()];
int count = Chars.split(text, delimiter, buffer);
if (count == buffer.length) {
return buffer;
}
Chars[] result = new Chars[count];
System.arraycopy(buffer, 0, result, 0, count);
return result;
}
public static String[] split(String text, String delimiter) {
E.checkArgument(delimiter.length() > 0,
"The delimiter can't be empty");
Chars[] buffer = new Chars[text.length()];
int count = Chars.split(text, delimiter, buffer);
String[] result = new String[count];
for (int i = 0; i < count; i++) {
result[i] = buffer[i].toString();
}
return result;
}
public static class Chars implements CharSequence {
private final char[] chars;
private final int start;
private final int end;
public Chars(char[] chars, int start, int end) {
E.checkArgument(0 < start && start < chars.length || start == 0,
"Invalid start parameter %s", start);
E.checkArgument(start <= end && end <= chars.length,
"Invalid end parameter %s", end);
this.chars = chars;
this.start = start;
this.end = end;
}
@Override
public int length() {
return this.end - this.start;
}
@Override
public char charAt(int index) {
return this.chars[this.start + index];
}
@Override
public CharSequence subSequence(int start, int end) {
return new Chars(this.chars, this.start + start, this.start + end);
}
@Override
public boolean equals(Object object) {
if (!(object instanceof Chars)) {
return false;
}
Chars other = (Chars) object;
return this.toString().equals(other.toString());
}
@Override
public int hashCode() {
return this.toString().hashCode();
}
@Override
public String toString() {
return new String(this.chars, this.start, this.length());
}
public static Chars of(String string) {
return new Chars(string.toCharArray(), 0, string.length());
}
public static Chars[] of(String... strings) {
Chars[] results = new Chars[strings.length];
for (int i = 0; i < strings.length; i++) {
results[i] = Chars.of(strings[i]);
}
return results;
}
public static int split(String text, String delimiter, Chars[] buffer) {
int count = 0;
int from = 0;
char[] chars = text.toCharArray();
for (int to; (to = text.indexOf(delimiter, from)) >= 0;
from = to + delimiter.length()) {
buffer[count++] = new Chars(chars, from, to);
}
if (from < text.length()) {
buffer[count++] = new Chars(chars, from, text.length());
}
return count;
}
}
}

View File

@ -1,59 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.time.Duration;
import java.util.Date;
public final class TimeUtil {
@SuppressWarnings("deprecation")
public static long BASE_TIME = new Date(2017 - 1900, 10, 28).getTime();
public static long timeGen() {
return System.currentTimeMillis() - BASE_TIME;
}
public static long timeGen(Date date) {
return date.getTime() - BASE_TIME;
}
public static long timeGen(long time) {
return time - BASE_TIME;
}
public static long tillNextMillis(long lastTimestamp) {
long timestamp = timeGen();
while (timestamp <= lastTimestamp) {
timestamp = timeGen();
}
return timestamp;
}
public static String readableTime(long time) {
if (time > 60 * 1000) {
// Remove the milliseconds part
time = time / 1000 * 1000;
}
Duration duration = Duration.ofMillis(time);
return duration.toString()
.substring(2)
.replaceAll("(\\d[HMS])(?!$)", "$1 ")
.toLowerCase();
}
}

View File

@ -1,181 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.time.Duration;
public final class UnitUtil {
public static double bytesToMB(long bytes) {
return doubleWith2Scale(bytes / (double) Bytes.MB);
}
public static double bytesToGB(long bytes) {
return doubleWith2Scale(bytes / (double) Bytes.GB);
}
public static double doubleWith2Scale(double value) {
BigDecimal decimal = new BigDecimal(value);
return decimal.setScale(2, RoundingMode.HALF_UP).doubleValue();
}
public static String bytesToReadableString(long bytes) {
// NOTE: FileUtils.byteCountToDisplaySize() lost decimal precision
final String[] units = {"B", "KB", "MB", "GB", "TB", "PB", "EB"};
if (bytes <= 0L) {
return "0 B";
}
int i = (int) (Math.log(bytes) / Math.log(1024));
E.checkArgument(i < units.length,
"The bytes parameter is out of %s unit: %s",
units[units.length - 1], bytes);
double value = bytes / Math.pow(1024, i);
if (value % 1L == 0L) {
return ((long) value) + " " + units[i];
} else {
return doubleWith2Scale(value) + " " + units[i];
}
}
public static long bytesFromReadableString(String valueWithUnit) {
int spacePos = valueWithUnit.indexOf(" ");
E.checkArgument(spacePos >= 0,
"Invalid readable bytes '%s', " +
"expect format like '10 MB'", valueWithUnit);
String unit = valueWithUnit.substring(spacePos + 1);
long factor;
switch (unit.trim().toUpperCase()) {
case "B":
case "BYTE":
case "BYTES":
factor = 1L;
break;
case "KB":
case "KIB":
factor = Bytes.KB;
break;
case "MB":
case "MIB":
factor = Bytes.MB;
break;
case "GB":
case "GIB":
factor = Bytes.GB;
break;
case "TB":
case "TIB":
factor = Bytes.TB;
break;
case "PB":
case "PIB":
factor = Bytes.PB;
break;
case "EB":
case "EIB":
factor = Bytes.EB;
break;
default:
throw new IllegalArgumentException("Unrecognized unit " + unit);
}
double value;
try {
value = Double.parseDouble(valueWithUnit.substring(0, spacePos));
} catch (Exception e) {
throw new IllegalArgumentException(String.format(
"Invalid parameter(not number): '%s'", valueWithUnit), e);
}
value = value * factor;
E.checkArgument(value <= Long.MAX_VALUE,
"The value %s from parameter '%s' is out of range",
value, valueWithUnit);
return (long) value;
}
public static String timestampToReadableString(long time) {
Duration duration = Duration.ofMillis(time);
long days = duration.toDays();
long hours = duration.toHours();
long minutes = duration.toMinutes();
long seconds = duration.getSeconds();
if (days > 0) {
return String.format("%dd%dh%dm%ds",
days,
hours % 24,
minutes % 60,
seconds % 60);
} else if (hours > 0) {
return String.format("%dh%dm%ds",
hours,
minutes % 60,
seconds % 60);
} else if (minutes > 0) {
return String.format("%dm%ds",
minutes,
seconds % 60);
} else if (seconds > 0) {
long ms = duration.toMillis() % 1000L;
if (ms > 0L) {
return String.format("%ds%dms", seconds, ms);
} else {
return String.format("%ds", seconds);
}
} else {
return String.format("%dms", duration.toMillis());
}
}
public static long timestampFromReadableString(String valueWithUnit) {
long ms = 0L;
// Adapt format 'nDnHnMnS' to 'PnYnMnDTnHnMnS'
String formatDuration = valueWithUnit.toUpperCase();
if (formatDuration.indexOf('D') >= 0) {
// Contains days
assert !formatDuration.contains("MS");
formatDuration = "P" + formatDuration.replace("D", "DT");
} else {
// Not exists days
int msPos = formatDuration.indexOf("MS");
// If contains ms, remove the ms part
if (msPos >= 0) {
int sPos = formatDuration.indexOf("S");
if (0 <= sPos && sPos < msPos) {
// If contains second part
sPos += 1;
ms = Long.parseLong(formatDuration.substring(sPos, msPos));
ms %= 1000L;
formatDuration = formatDuration.substring(0, sPos);
} else {
// Not contains second part, only exists ms
ms = Long.parseLong(formatDuration.substring(0, msPos));
return ms;
}
} else {
assert formatDuration.endsWith("S");
}
formatDuration = "PT" + formatDuration;
}
Duration duration = Duration.parse(formatDuration);
return duration.toMillis() + ms;
}
}

View File

@ -1,236 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.util;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.util.Objects;
import java.util.jar.Attributes;
import java.util.jar.Manifest;
public final class VersionUtil {
/**
* Compare if a version is inside a range [begin, end)
* @param version The version to be compared
* @param begin The lower bound of the range
* @param end The upper bound of the range
* @return true if belong to the range, otherwise false
*/
public static boolean match(Version version, String begin, String end) {
E.checkArgumentNotNull(version, "The version to match is null");
return version.compareTo(new Version(begin)) >= 0 &&
version.compareTo(new Version(end)) < 0;
}
/**
* Compare if a version is greater than the other one (inclusive)
* @param version The version to be compared
* @param other The lower bound of the range
* @return true if it's greater than the other, otherwise false
*/
public static boolean gte(String version, String other) {
E.checkArgumentNotNull(version, "The version to match is null");
return new Version(version).compareTo(new Version(other)) >= 0;
}
/**
* Check whether a component version is matched expected range,
* throw an exception if it's not matched.
* @param version The version to be checked
* @param begin The lower bound of the range
* @param end The upper bound of the range
* @param component The owner component of version
*/
public static void check(Version version, String begin, String end,
String component) {
E.checkState(VersionUtil.match(version, begin, end),
"The version %s of '%s' is not in [%s, %s)",
version, component, begin, end);
}
/**
* Get implementation version from manifest in jar
* @param clazz The class to be load from jar package
* @return The implementation version
*/
public static String getImplementationVersion(Class<?> clazz) {
/*
* We don't use Package.getImplementationVersion() due to
* a duplicate package would override the origin package info.
*/
String className = clazz.getSimpleName() + ".class";
String classPath = Objects.requireNonNull(clazz.getResource(className)).toString();
if (!classPath.startsWith("jar:file:")) {
// Class not from JAR
return null;
}
int offset = classPath.lastIndexOf("!");
assert offset > 0;
// Get manifest file path
String manifestPath = classPath.substring(0, offset + 1);
return getImplementationVersion(manifestPath);
}
public static String getImplementationVersion(String manifestPath) {
manifestPath += "/META-INF/MANIFEST.MF";
Manifest manifest;
try {
manifest = new Manifest(new URL(manifestPath).openStream());
} catch (IOException ignored) {
return null;
}
return manifest.getMainAttributes()
.getValue(Attributes.Name.IMPLEMENTATION_VERSION);
}
/**
* Get version from pom.xml
* @return The pom version
*/
public static String getPomVersion() {
String cmd = "mvn help:evaluate -Dexpression=project.version " +
"-q -DforceStdout";
Process process = null;
InputStreamReader isr = null;
try {
process = Runtime.getRuntime().exec(cmd);
process.waitFor();
isr = new InputStreamReader(process.getInputStream());
BufferedReader br = new BufferedReader(isr);
return br.readLine();
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
if (isr != null) {
try {
isr.close();
} catch (Exception ignored) {
// pass
}
}
// Destroy child process
if (process != null) {
process.destroy();
}
}
}
public static class Version implements Comparable<Version> {
public static Version of(Class<?> clazz) {
return Version.of(clazz, null);
}
public static Version of(Class<?> clazz, String defaultValue) {
String v = getImplementationVersion(clazz);
if (v == null) {
v = defaultValue;
}
return v == null ? null : new Version(v);
}
public static Version of(String version) {
return new Version(version);
}
/************************** Version define **************************/
private final String version;
private final int[] parts;
public Version(String version) {
E.checkArgumentNotNull(version, "The version is null");
E.checkArgument(version.matches("[0-9]+(\\.[0-9]+)*"),
"Invalid version format: %s", version);
this.version = version;
this.parts = parseVersion(version);
}
private static int[] parseVersion(String version) {
String[] parts = version.split("\\.");
int[] partsNumber = new int[parts.length];
for (int i = 0; i < parts.length; i++) {
partsNumber[i] = Integer.parseInt(parts[i]);
}
return partsNumber;
}
public final String get() {
return this.version;
}
@Override
public int compareTo(Version that) {
if (that == null) {
return 1;
}
int[] thisParts = this.parts;
int[] thatParts = that.parts;
int length = Math.max(thisParts.length, thatParts.length);
for (int i = 0; i < length; i++) {
int thisPart = i < thisParts.length ? thisParts[i] : 0;
int thatPart = i < thatParts.length ? thatParts[i] : 0;
if (thisPart < thatPart) {
return -1;
}
if (thisPart > thatPart) {
return 1;
}
}
return 0;
}
@Override
public boolean equals(Object that) {
if (this == that) {
return true;
}
if (that == null) {
return false;
}
if (this.getClass() != that.getClass()) {
return false;
}
return this.compareTo((Version) that) == 0;
}
@Override
public int hashCode() {
int hash = 0;
for (int i = this.parts.length - 1; i >= 0; i--) {
int part = this.parts[i];
if (part == 0 && hash == 0) {
continue;
}
hash = 31 * hash + Integer.hashCode(part);
}
return hash;
}
@Override
public String toString() {
return this.version;
}
}
}

View File

@ -1,28 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.version;
import org.apache.hugegraph.util.VersionUtil.Version;
public class CommonVersion {
public static final String NAME = "hugegraph-common";
// The second parameter of Version.of() is for all-in-one JAR
public static final Version VERSION = Version.of(CommonVersion.class, "1.5.0");
}

View File

@ -1,444 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.testutil;
import org.apache.hugegraph.unit.BaseUnitTest;
import org.junit.Test;
public class AssertTest extends BaseUnitTest {
@Test
public void testAssertEquals() {
Assert.assertEquals((byte) 1, Byte.valueOf("1"));
Assert.assertEquals((short) 1, Short.valueOf("1"));
Assert.assertEquals('1', Character.valueOf('1'));
Assert.assertEquals(1, Integer.valueOf("1"));
Assert.assertEquals(1L, Long.valueOf("1"));
Assert.assertEquals(1f, Float.valueOf("1"));
Assert.assertEquals(1d, Double.valueOf("1"));
}
@Test
public void testAssertEqualsWithError() {
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals((byte) 1, "1");
}, e -> {
Assert.assertContains("expected: java.lang.Byte",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals((short) 1, "1");
}, e -> {
Assert.assertContains("expected: java.lang.Short",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals('1', "1");
}, e -> {
Assert.assertContains("expected: java.lang.Character",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1, "1");
}, e -> {
Assert.assertContains("expected: java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1L, "1");
}, e -> {
Assert.assertContains("expected: java.lang.Long",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1f, "1.0");
}, e -> {
Assert.assertContains("expected: java.lang.Float",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1d, "1.0");
}, e -> {
Assert.assertContains("expected: java.lang.Double",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1f, "1");
}, e -> {
Assert.assertContains("expected:<1.0> but was:<1>",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1d, "1");
}, e -> {
Assert.assertContains("expected:<1.0> but was:<1>",
e.getMessage());
});
}
@Test
public void testAssertEqualsOfIntWithError() {
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1, (Byte) (byte) 1);
}, e -> {
Assert.assertContains("expected: java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1, (Short) (short) 1);
}, e -> {
Assert.assertContains("expected: java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1, (Character) '1');
}, e -> {
Assert.assertContains("expected: java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1, (Long) 1L);
}, e -> {
Assert.assertContains("expected: java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1, (Float) 1f);
}, e -> {
Assert.assertContains("expected:<1> but was:<1.0>",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1, (Double) 1d);
}, e -> {
Assert.assertContains("expected:<1> but was:<1.0>",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1, "1.0");
}, e -> {
Assert.assertContains("expected:<1> but was:<1.0>",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1, (Byte) (byte) 2);
}, e -> {
Assert.assertContains("expected:<1> but was:<2>",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertEquals(1, null);
}, e -> {
Assert.assertContains("expected:<1> but was:<null>",
e.getMessage());
});
}
@Test
public void testAssertThrows() {
Assert.assertThrows(NullPointerException.class, () -> {
throw new NullPointerException();
});
Assert.assertThrows(RuntimeException.class, () -> {
throw new RuntimeException();
});
Throwable exception = Assert.assertThrows(RuntimeException.class, () -> {
throw new RuntimeException("fake-error");
});
Assert.assertInstanceOf(RuntimeException.class, exception);
Assert.assertEquals("fake-error", exception.getMessage());
Assert.assertThrows(RuntimeException.class, () -> {
throw new RuntimeException("fake-error");
}, e -> {
Assert.assertEquals("fake-error", e.getMessage());
});
}
@Test
public void testAssertThrowsWithTypeError() {
try {
Assert.assertThrows(NullPointerException.class, () -> {
// pass
});
Assert.fail("Expect error");
} catch (AssertionError e) {
Assert.assertContains("java.lang.NullPointerException", e.getMessage());
}
try {
Assert.assertThrows(NullPointerException.class, () -> {
throw new RuntimeException();
});
Assert.fail("Expect error");
} catch (AssertionError e) {
Assert.assertContains("java.lang.NullPointerException", e.getMessage());
Assert.assertContains("java.lang.RuntimeException", e.getMessage());
}
}
@Test
public void testAssertThrowsWithMessageError() {
try {
Assert.assertThrows(RuntimeException.class, () -> {
throw new RuntimeException("fake-error");
}, e -> {
Assert.assertEquals("fake-error-typo", e.getMessage());
});
Assert.fail("Expect error");
} catch (AssertionError e) {
Assert.assertContains("expected:<fake-error[-typo]> but was:<fake-error[]>",
e.getMessage());
}
}
@Test
public void testAssertGt() {
Assert.assertGt((byte) 1, Byte.valueOf("2"));
Assert.assertGt((short) 1, Short.valueOf("2"));
Assert.assertGt(1, Integer.valueOf("2"));
Assert.assertGt(1L, Long.valueOf("2"));
Assert.assertGt(1f, Float.valueOf("1.01"));
Assert.assertGt(1d, Double.valueOf("1.01"));
Assert.assertGt((byte) 1, (byte) 2);
Assert.assertGt((short) 1, (short) 2);
Assert.assertGt(1, 2);
Assert.assertGt(1L, 2L);
Assert.assertGt(1f, 1.01f);
Assert.assertGt(1d, 1.01d);
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGt(1, 0);
}, e -> {
Assert.assertContains("Expected: a number > 1", e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGt(1, null);
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGt(1, (byte) 2);
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGt(1, 1.1);
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGt(1, '2');
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGt(0.9, 1);
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Double",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGt(0.9d, 0.98f);
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Double",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGt(0.9f, 0.98d);
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Float",
e.getMessage());
});
}
@Test
public void testAssertGte() {
Assert.assertGte((byte) 1, Byte.valueOf("2"));
Assert.assertGte((short) 1, Short.valueOf("2"));
Assert.assertGte(1, Integer.valueOf("2"));
Assert.assertGte(1L, Long.valueOf("2"));
Assert.assertGte(1f, Float.valueOf("1.01"));
Assert.assertGte(1d, Double.valueOf("1.01"));
Assert.assertGte((byte) 1, Byte.valueOf("1"));
Assert.assertGte((short) 1, Short.valueOf("1"));
Assert.assertGte(1, Integer.valueOf("1"));
Assert.assertGte(1L, Long.valueOf("1"));
Assert.assertGte(1f, Float.valueOf("1"));
Assert.assertGte(1d, Double.valueOf("1"));
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGte(1, 0);
}, e -> {
Assert.assertContains("Expected: a number >= 1", e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGte(1, 1.1);
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGte(1, '2');
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Integer",
e.getMessage());
});
}
@Test
public void testAssertLt() {
Assert.assertLt((byte) 1, Byte.valueOf("0"));
Assert.assertLt((short) 1, Short.valueOf("0"));
Assert.assertLt(1, Integer.valueOf("0"));
Assert.assertLt(1L, Long.valueOf("0"));
Assert.assertLt(1f, Float.valueOf("0.99"));
Assert.assertLt(1d, Double.valueOf("0.99"));
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertLt(1, 2);
}, e -> {
Assert.assertContains("Expected: a number < 1", e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGt(1, 0.9);
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertGt(1, '0');
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Integer",
e.getMessage());
});
}
@Test
public void testAssertLte() {
Assert.assertLte((byte) 1, Byte.valueOf("0"));
Assert.assertLte((short) 1, Short.valueOf("0"));
Assert.assertLte(1, Integer.valueOf("0"));
Assert.assertLte(1L, Long.valueOf("0"));
Assert.assertLte(1f, Float.valueOf("0.99"));
Assert.assertLte(1d, Double.valueOf("0.99"));
Assert.assertLte((byte) 1, Byte.valueOf("1"));
Assert.assertLte((short) 1, Short.valueOf("1"));
Assert.assertLte(1, Integer.valueOf("1"));
Assert.assertLte(1L, Long.valueOf("1"));
Assert.assertLte(1f, Float.valueOf("1"));
Assert.assertLte(1d, Double.valueOf("1"));
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertLte(1, 2);
}, e -> {
Assert.assertContains("Expected: a number <= 1", e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertLte(1, 0.9);
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Integer",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertLte(1, '0');
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Integer",
e.getMessage());
});
}
@Test
public void testAssertContains() {
Assert.assertContains("test", "test");
Assert.assertContains("test", "hellotest");
Assert.assertContains("test", "test123");
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertContains("test123", "test");
}, e -> {
Assert.assertContains("Expected: a string containing",
e.getMessage());
});
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertContains("null", null);
}, e -> {
Assert.assertContains("Expected: a string containing",
e.getMessage());
});
Assert.assertThrows(NullPointerException.class, () -> {
Assert.assertContains(null, "null");
}, e -> {
Assert.assertNull(e.getMessage());
});
}
@Test
public void testAssertInstanceOf() {
Assert.assertInstanceOf(Integer.class, 1);
Assert.assertInstanceOf(Double.class, 1.0);
Assert.assertInstanceOf(String.class, "1.0");
Assert.assertInstanceOf(BaseUnitTest.class, this);
Assert.assertThrows(AssertionError.class, () -> {
Assert.assertInstanceOf(Float.class, 1);
}, e -> {
Assert.assertContains("Expected: an instance of java.lang.Float",
e.getMessage());
});
}
}

View File

@ -1,258 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.testutil;
import org.junit.Test;
public class WhiteboxTest {
@Test
public void testGetStaticInternalState() {
Assert.assertEquals(1, Whitebox.getInternalState(Test1.class,
"staticValue"));
Test1 test1 = newTest();
Assert.assertEquals(1, Whitebox.getInternalState(test1, "staticValue"));
Assert.assertEquals(2, Whitebox.getInternalState(test1,
"test2.staticValue"));
}
@Test
public void testSetStaticInternalState() {
try {
Whitebox.setInternalState(Test1.class, "staticValue", 11);
Assert.assertEquals(11, Test1.staticValue);
Test1 test1 = newTest();
Whitebox.setInternalState(test1, "staticValue", 111);
Assert.assertEquals(111, Test1.staticValue);
Whitebox.setInternalState(test1, "test2.staticValue", 22);
Assert.assertEquals(22, Test2.staticValue);
} finally {
Whitebox.setInternalState(Test1.class, "staticValue", 1);
Whitebox.setInternalState(Test2.class, "staticValue", 2);
}
}
@Test
public void testGetInternalState() {
Test1 test1 = newTest();
Assert.assertEquals(1, Whitebox.getInternalState(test1, "ivalue"));
Assert.assertEquals(2f, Whitebox.getInternalState(test1,
"test2.finalValue"));
Assert.assertEquals("3", Whitebox.getInternalState(test1,
"test2.test3.str"));
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.getInternalState(test1, "ivalue2");
});
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.getInternalState(test1, "test2.valueNotExist");
});
}
@Test
public void testSetInternalState() {
Test1 test1 = newTest();
Whitebox.setInternalState(test1, "ivalue", 11);
Assert.assertEquals(11, Whitebox.getInternalState(test1, "ivalue"));
Assert.assertEquals(11, test1.ivalue);
Whitebox.setInternalState(test1, "test2.finalValue", 22f);
Assert.assertEquals(22f, Whitebox.getInternalState(test1,
"test2.finalValue"));
Whitebox.setInternalState(test1, "test2.test3.str", "33");
Assert.assertEquals("33", Whitebox.getInternalState(test1,
"test2.test3.str"));
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.setInternalState(test1, "ivalue2", 11);
});
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.setInternalState(test1, "test2.valueNotExist", 22f);
});
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.setInternalState(test1, "test2.finalValue", 22d);
});
Assert.assertThrows(RuntimeException.class, () -> {
test1.test2 = null;
Whitebox.setInternalState(test1, "test2.finalValue", 22f);
}, e -> {
Assert.assertContains("Can't set value on null field",
e.getMessage());
});
}
@Test
public void testSetInternalFinalState() {
Test1 test1 = newTest();
Assert.assertEquals(1, test1.finalValue);
Whitebox.setInternalState(test1, "finalValue", 2);
Assert.assertEquals(2, Whitebox.getInternalState(test1, "finalValue"));
Whitebox.setInternalState(test1, "finalValue", 3);
Assert.assertEquals(3, Whitebox.getInternalState(test1, "finalValue"));
Whitebox.setInternalState(test1, "test2.finalValue", 22f);
Assert.assertEquals(22f, Whitebox.getInternalState(test1,
"test2.finalValue"));
// FIXME: seems don't take effect!!!
Assert.assertEquals(1, test1.finalValue);
Assert.assertEquals(2f, test1.test2.finalValue, 0f);
}
@Test
public void testInvokeStatic() {
Assert.assertEquals(1, Whitebox.invokeStatic(Test1.class, "svalue"));
Assert.assertEquals(2, Whitebox.invokeStatic(Test1.class, "svalue", 2));
Assert.assertEquals(2, Whitebox.invokeStatic(Test1.class, "svalue",
new Integer(2)));
Assert.assertEquals(2d, Whitebox.invokeStatic(Test1.class,
new Class[]{Object.class},
"svalue", 2d));
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.invokeStatic(Test1.class, "svalue2");
});
Assert.assertThrows(IllegalArgumentException.class, () -> {
Whitebox.invokeStatic(Test1.class, "throwfunc1");
});
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.invokeStatic(Test1.class, "throwfunc2");
});
}
@Test
public void testInvoke() {
Test1 test1 = newTest();
Assert.assertEquals(1, Whitebox.invoke(test1.getClass(),
"value", test1));
Assert.assertEquals(3, Whitebox.invoke(test1.getClass(),
"addValue", test1, 2));
Assert.assertEquals(2f, Whitebox.invoke(test1, "test2", "value"));
Assert.assertEquals(2, Whitebox.invoke(test1, "test2",
new Class[]{Object.class},
"value", 2));
Assert.assertEquals(3, Whitebox.invoke(test1, "test4", "addValue", 2));
Assert.assertEquals(4, Whitebox.invoke(test1, "test4", "value"));
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.invoke(test1.getClass(), "value2", test1);
});
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.invoke(test1, "test22", "value");
});
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.invoke(test1, "test2", "value", 2);
});
Assert.assertThrows(RuntimeException.class, () -> {
Whitebox.invoke(test1.getClass(), "addValue", test1, 2.0);
});
}
private static Test1 newTest() {
Test1 test1 = new Test1();
test1.test2 = new Test2();
test1.test2.test3 = new Test3();
test1.test4 = new TestSubClass();
return test1;
}
@SuppressWarnings("unused")
private static class Test1 {
private static int staticValue = 1;
private int ivalue = 1;
private final int finalValue = 1;
private Test2 test2;
private TestSubClass test4;
private int value() {
return this.ivalue;
}
private int addValue(int i) {
return this.ivalue + i;
}
private static int svalue() {
return 1;
}
private static int svalue(int i) {
return i;
}
private static <T> T svalue(T o) {
return o;
}
private static int throwfunc1() {
throw new IllegalArgumentException("fake runtime exception");
}
private static int throwfunc2() throws Exception {
throw new Exception("fake exception");
}
}
@SuppressWarnings("unused")
private static class Test2 {
private static int staticValue = 2;
private final float finalValue = 2f;
private Test3 test3;
private float value() {
return this.finalValue;
}
private <T> T value(T o) {
return o;
}
}
@SuppressWarnings("unused")
private static class Test3 {
private String str = "3";
private String value() {
return this.str;
}
}
@SuppressWarnings("unused")
private static class TestSubClass extends Test1 {
private int value() {
return 4;
}
}
}

View File

@ -1,71 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.unit;
import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import org.apache.commons.io.FileUtils;
import org.apache.hugegraph.util.ExceptionUtil;
import org.apache.hugegraph.util.TimeUtil;
import org.junit.AfterClass;
import org.junit.BeforeClass;
public class BaseUnitTest {
@BeforeClass
public static void init() {
// pass
}
@AfterClass
public static void clear() throws Exception {
// pass
}
protected static void runWithThreads(int threads, Runnable task) {
ExecutorService executor = Executors.newFixedThreadPool(threads);
List<Future<?>> futures = new ArrayList<>();
for (int i = 0; i < threads; i++) {
futures.add(executor.submit(task));
}
for (Future<?> future : futures) {
ExceptionUtil.futureGet(future);
}
}
protected static void waitTillNext(long seconds) {
TimeUtil.tillNextMillis(TimeUtil.timeGen() + seconds * 1000);
}
public static void downloadFileByUrl(String url, String destPath) {
int connectTimeout = 5000;
int readTimeout = 5000;
try {
FileUtils.copyURLToFile(new URL(url), new File(destPath), connectTimeout, readTimeout);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}

View File

@ -1,127 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.unit;
import org.apache.hugegraph.testutil.AssertTest;
import org.apache.hugegraph.testutil.WhiteboxTest;
import org.apache.hugegraph.unit.config.HugeConfigTest;
import org.apache.hugegraph.unit.config.OptionSpaceTest;
import org.apache.hugegraph.unit.event.EventHubTest;
import org.apache.hugegraph.unit.rest.AbstractRestClientTest;
import org.apache.hugegraph.unit.version.VersionTest;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.apache.hugegraph.unit.concurrent.AtomicLockTest;
import org.apache.hugegraph.unit.concurrent.BarrierEventTest;
import org.apache.hugegraph.unit.concurrent.KeyLockTest;
import org.apache.hugegraph.unit.concurrent.LockGroupTest;
import org.apache.hugegraph.unit.concurrent.LockManagerTest;
import org.apache.hugegraph.unit.concurrent.PausableScheduledThreadPoolTest;
import org.apache.hugegraph.unit.concurrent.RowLockTest;
import org.apache.hugegraph.unit.date.SafeDateFormatTest;
import org.apache.hugegraph.unit.iterator.BatchMapperIteratorTest;
import org.apache.hugegraph.unit.iterator.ExtendableIteratorTest;
import org.apache.hugegraph.unit.iterator.FilterIteratorTest;
import org.apache.hugegraph.unit.iterator.FlatMapperFilterIteratorTest;
import org.apache.hugegraph.unit.iterator.FlatMapperIteratorTest;
import org.apache.hugegraph.unit.iterator.LimitIteratorTest;
import org.apache.hugegraph.unit.iterator.ListIteratorTest;
import org.apache.hugegraph.unit.iterator.MapperIteratorTest;
import org.apache.hugegraph.unit.license.LicenseExtraParamTest;
import org.apache.hugegraph.unit.license.LicenseCreateParamTest;
import org.apache.hugegraph.unit.license.LicenseInstallParamTest;
import org.apache.hugegraph.unit.license.LicenseParamsTest;
import org.apache.hugegraph.unit.license.MachineInfoTest;
import org.apache.hugegraph.unit.perf.PerfUtilTest;
import org.apache.hugegraph.unit.perf.StopwatchTest;
import org.apache.hugegraph.unit.rest.RestClientTest;
import org.apache.hugegraph.unit.rest.RestResultTest;
import org.apache.hugegraph.unit.util.BytesTest;
import org.apache.hugegraph.unit.util.CollectionUtilTest;
import org.apache.hugegraph.unit.util.DateUtilTest;
import org.apache.hugegraph.unit.util.EcheckTest;
import org.apache.hugegraph.unit.util.HashUtilTest;
import org.apache.hugegraph.unit.util.InsertionOrderUtilTest;
import org.apache.hugegraph.unit.util.LogTest;
import org.apache.hugegraph.unit.util.LongEncodingTest;
import org.apache.hugegraph.unit.util.NumericUtilTest;
import org.apache.hugegraph.unit.util.OrderLimitMapTest;
import org.apache.hugegraph.unit.util.ReflectionUtilTest;
import org.apache.hugegraph.unit.util.StringUtilTest;
import org.apache.hugegraph.unit.util.TimeUtilTest;
import org.apache.hugegraph.unit.util.UnitUtilTest;
import org.apache.hugegraph.unit.util.VersionUtilTest;
@RunWith(Suite.class)
@Suite.SuiteClasses({
LockManagerTest.class,
LockGroupTest.class,
AtomicLockTest.class,
KeyLockTest.class,
RowLockTest.class,
PausableScheduledThreadPoolTest.class,
HugeConfigTest.class,
OptionSpaceTest.class,
SafeDateFormatTest.class,
BarrierEventTest.class,
EventHubTest.class,
PerfUtilTest.class,
StopwatchTest.class,
AbstractRestClientTest.class,
RestClientTest.class,
RestResultTest.class,
VersionTest.class,
ExtendableIteratorTest.class,
FilterIteratorTest.class,
LimitIteratorTest.class,
MapperIteratorTest.class,
FlatMapperIteratorTest.class,
FlatMapperFilterIteratorTest.class,
ListIteratorTest.class,
BatchMapperIteratorTest.class,
BytesTest.class,
CollectionUtilTest.class,
EcheckTest.class,
HashUtilTest.class,
InsertionOrderUtilTest.class,
LogTest.class,
NumericUtilTest.class,
ReflectionUtilTest.class,
StringUtilTest.class,
TimeUtilTest.class,
VersionUtilTest.class,
LongEncodingTest.class,
OrderLimitMapTest.class,
DateUtilTest.class,
UnitUtilTest.class,
LicenseExtraParamTest.class,
LicenseCreateParamTest.class,
LicenseInstallParamTest.class,
LicenseParamsTest.class,
MachineInfoTest.class,
AssertTest.class,
WhiteboxTest.class
})
public class UnitTestSuite {
}

View File

@ -1,62 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.unit.concurrent;
import org.junit.Test;
import org.apache.hugegraph.concurrent.AtomicLock;
import org.apache.hugegraph.testutil.Assert;
import org.apache.hugegraph.unit.BaseUnitTest;
public class AtomicLockTest extends BaseUnitTest {
@Test
public void testLockUnlock() {
AtomicLock lock = new AtomicLock("lock");
Assert.assertEquals("lock", lock.name());
Assert.assertTrue(lock.lock(0));
try {
Assert.assertFalse(lock.lock(1));
// lock in other threads
runWithThreads(2, () -> {
Assert.assertFalse(lock.tryLock());
});
lock.unlock();
} finally {
lock.unlock();
// unlock multi times is OK
lock.unlock();
lock.unlock();
}
Assert.assertThrows(RuntimeException.class, () -> {
lock.lock(-1);
}, e -> {
Assert.assertContains("Locking retry times should be in [0, 10], " +
"but got -1", e.getMessage());
});
Assert.assertThrows(RuntimeException.class, () -> {
lock.lock(11);
}, e -> {
Assert.assertContains("Locking retry times should be in [0, 10], " +
"but got 11", e.getMessage());
});
}
}

View File

@ -1,288 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.unit.concurrent;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import org.junit.Test;
import org.apache.hugegraph.concurrent.BarrierEvent;
import org.apache.hugegraph.testutil.Assert;
public class BarrierEventTest {
private static final int WAIT_THREADS_COUNT = 10;
@Test(timeout = 5000)
public void testAWait() throws InterruptedException {
BarrierEvent barrierEvent = new BarrierEvent();
AtomicInteger result = new AtomicInteger(0);
CountDownLatch latch = new CountDownLatch(2);
Thread awaitThread = new Thread(() -> {
try {
barrierEvent.await();
result.incrementAndGet();
} catch (InterruptedException e) {
// Do nothing.
} finally {
latch.countDown();
}
});
awaitThread.start();
Thread signalThread = new Thread(() -> {
barrierEvent.signalAll();
latch.countDown();
});
signalThread.start();
latch.await();
Assert.assertEquals(1, result.get());
}
@Test
public void testAWaitWithTimeout() throws InterruptedException {
BarrierEvent barrierEvent = new BarrierEvent();
boolean signaled = barrierEvent.await(1L);
Assert.assertFalse(signaled);
}
@Test
public void testReset() throws InterruptedException {
BarrierEvent barrierEvent = new BarrierEvent();
boolean signaled = barrierEvent.await(1L);
Assert.assertFalse(signaled);
barrierEvent.signal();
signaled = barrierEvent.await(1L);
Assert.assertTrue(signaled);
barrierEvent.reset();
signaled = barrierEvent.await(1L);
Assert.assertFalse(signaled);
}
@Test
public void testSignal() throws InterruptedException {
BarrierEvent barrierEvent = new BarrierEvent();
boolean signaled = barrierEvent.await(1L);
Assert.assertFalse(signaled);
barrierEvent.signal();
signaled = barrierEvent.await(1L);
Assert.assertTrue(signaled);
}
@Test
public void testSignalByMultiThreadWithSignalFirst()
throws InterruptedException {
BarrierEvent barrierEvent = new BarrierEvent();
AtomicInteger eventCount = new AtomicInteger(0);
AtomicInteger waitThreadInterruptedCount = new AtomicInteger(0);
ExecutorService executorService =
Executors.newFixedThreadPool(WAIT_THREADS_COUNT + 1);
CountDownLatch waitLatch = new CountDownLatch(WAIT_THREADS_COUNT);
CountDownLatch signalLatch = new CountDownLatch(1);
for (int i = 0; i < WAIT_THREADS_COUNT; i++) {
executorService.submit(() -> {
try {
signalLatch.await();
barrierEvent.await();
eventCount.incrementAndGet();
} catch (InterruptedException e) {
waitThreadInterruptedCount.incrementAndGet();
} finally {
waitLatch.countDown();
}
});
}
executorService.submit(() -> {
barrierEvent.signal();
signalLatch.countDown();
});
executorService.shutdown();
executorService.awaitTermination(2L, TimeUnit.SECONDS);
waitLatch.await();
Assert.assertEquals(10, eventCount.get());
Assert.assertEquals(0, waitThreadInterruptedCount.get());
}
@Test
public void testSignalByMultiThreadWithSignalLast()
throws InterruptedException {
BarrierEvent barrierEvent = new BarrierEvent();
AtomicInteger eventCount = new AtomicInteger(0);
AtomicInteger waitThreadInterruptedCount = new AtomicInteger(0);
AtomicInteger signalThreadInterruptedCount = new AtomicInteger(0);
ExecutorService executorService =
Executors.newFixedThreadPool(WAIT_THREADS_COUNT + 1);
CountDownLatch waitLatch = new CountDownLatch(WAIT_THREADS_COUNT);
CountDownLatch signalLatch = new CountDownLatch(1);
for (int i = 0; i < WAIT_THREADS_COUNT; i++) {
executorService.submit(() -> {
try {
waitLatch.countDown();
barrierEvent.await();
eventCount.incrementAndGet();
} catch (InterruptedException e) {
waitThreadInterruptedCount.incrementAndGet();
}
});
}
executorService.submit(() -> {
try {
waitLatch.await();
} catch (InterruptedException e) {
signalThreadInterruptedCount.incrementAndGet();
}
barrierEvent.signal();
signalLatch.countDown();
});
signalLatch.await();
executorService.shutdownNow();
executorService.awaitTermination(1L, TimeUnit.SECONDS);
Assert.assertEquals(1, eventCount.get());
Assert.assertEquals(WAIT_THREADS_COUNT - 1,
waitThreadInterruptedCount.get());
Assert.assertEquals(0, signalThreadInterruptedCount.get());
}
@Test
public void testSignalAll() throws InterruptedException {
BarrierEvent barrierEvent = new BarrierEvent();
boolean signaled = barrierEvent.await(1L);
Assert.assertFalse(signaled);
barrierEvent.signalAll();
signaled = barrierEvent.await(1L);
Assert.assertTrue(signaled);
}
@Test
public void testSignalAllByMultiThreadWithSignalFirst()
throws InterruptedException {
BarrierEvent barrierEvent = new BarrierEvent();
AtomicInteger eventCount = new AtomicInteger(0);
AtomicInteger waitThreadInterruptedCount = new AtomicInteger(0);
ExecutorService executorService =
Executors.newFixedThreadPool(WAIT_THREADS_COUNT + 1);
CountDownLatch waitLatch = new CountDownLatch(WAIT_THREADS_COUNT);
CountDownLatch signalLatch = new CountDownLatch(1);
for (int i = 0; i < WAIT_THREADS_COUNT; i++) {
executorService.submit(() -> {
try {
signalLatch.await();
waitLatch.countDown();
barrierEvent.await();
eventCount.incrementAndGet();
} catch (InterruptedException e) {
waitThreadInterruptedCount.incrementAndGet();
}
});
}
executorService.submit(() -> {
barrierEvent.signalAll();
signalLatch.countDown();
});
executorService.shutdown();
executorService.awaitTermination(1L, TimeUnit.SECONDS);
Assert.assertEquals(10, eventCount.get());
Assert.assertEquals(0, waitThreadInterruptedCount.get());
}
@Test
public void testSignalAllByMultiThreadWithSignalLast()
throws InterruptedException {
BarrierEvent barrierEvent = new BarrierEvent();
AtomicInteger eventCount = new AtomicInteger(0);
AtomicInteger waitThreadInterruptedCount = new AtomicInteger(0);
AtomicInteger signalThreadInterruptedCount = new AtomicInteger(0);
ExecutorService executorService =
Executors.newFixedThreadPool(WAIT_THREADS_COUNT + 1);
CountDownLatch waitLatch = new CountDownLatch(WAIT_THREADS_COUNT);
CountDownLatch signalLatch = new CountDownLatch(1);
for (int i = 0; i < WAIT_THREADS_COUNT; i++) {
executorService.submit(() -> {
try {
waitLatch.countDown();
barrierEvent.await();
eventCount.incrementAndGet();
} catch (InterruptedException e) {
waitThreadInterruptedCount.incrementAndGet();
}
});
}
executorService.submit(() -> {
try {
waitLatch.await();
} catch (InterruptedException e) {
signalThreadInterruptedCount.incrementAndGet();
}
barrierEvent.signalAll();
signalLatch.countDown();
});
signalLatch.await();
executorService.shutdown();
executorService.awaitTermination(1L, TimeUnit.SECONDS);
Assert.assertEquals(WAIT_THREADS_COUNT, eventCount.get());
Assert.assertEquals(0, waitThreadInterruptedCount.get());
Assert.assertEquals(0, signalThreadInterruptedCount.get());
}
@Test
public void testSignalAllByMultiThreadWithSignalAwaitConcurrent()
throws InterruptedException {
BarrierEvent barrierEvent = new BarrierEvent();
AtomicInteger eventCount = new AtomicInteger(0);
AtomicInteger waitThreadInterruptedCount = new AtomicInteger(0);
AtomicInteger signalThreadInterruptedCount = new AtomicInteger(0);
ExecutorService executorService =
Executors.newFixedThreadPool(WAIT_THREADS_COUNT + 1);
CountDownLatch syncLatch = new CountDownLatch(1);
for (int i = 0; i < WAIT_THREADS_COUNT; i++) {
executorService.submit(() -> {
try {
syncLatch.await();
barrierEvent.await();
eventCount.incrementAndGet();
} catch (InterruptedException e) {
waitThreadInterruptedCount.incrementAndGet();
}
});
}
executorService.submit(() -> {
try {
syncLatch.await();
} catch (InterruptedException e) {
signalThreadInterruptedCount.incrementAndGet();
}
barrierEvent.signalAll();
});
syncLatch.countDown();
executorService.shutdown();
executorService.awaitTermination(1L, TimeUnit.SECONDS);
Assert.assertEquals(WAIT_THREADS_COUNT, eventCount.get());
Assert.assertEquals(0, waitThreadInterruptedCount.get());
Assert.assertEquals(0, signalThreadInterruptedCount.get());
}
}

View File

@ -1,124 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.unit.concurrent;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.locks.Lock;
import org.junit.Test;
import org.apache.hugegraph.concurrent.KeyLock;
import org.apache.hugegraph.testutil.Assert;
import org.apache.hugegraph.unit.BaseUnitTest;
public class KeyLockTest extends BaseUnitTest {
@Test
public void testLockUnlock() {
KeyLock locks = new KeyLock();
locks.lock("1");
try {
// lock again is OK
locks.lock("1");
// lock in other threads
runWithThreads(1, () -> {
locks.lock("2");
});
locks.unlock("1");
} finally {
locks.unlock("1");
}
Assert.assertThrows(IllegalMonitorStateException.class, () -> {
locks.unlock("2");
});
Assert.assertThrows(IllegalMonitorStateException.class, () -> {
locks.unlock("3");
});
Assert.assertThrows(IllegalArgumentException.class, () -> {
locks.lock(null);
}, e -> {
Assert.assertContains("Lock key can't be null", e.getMessage());
});
Assert.assertThrows(IllegalArgumentException.class, () -> {
locks.unlock(null);
}, e -> {
Assert.assertContains("Unlock key can't be null", e.getMessage());
});
}
@Test
public void testLockUnlockAll() {
KeyLock locks = new KeyLock();
List<Lock> ls = locks.lockAll("1", 2);
locks.unlockAll(ls);
runWithThreads(1, () -> {
List<Lock> ls2 = locks.lockAll("1", 3);
locks.unlockAll(ls2);
});
List<Lock> ls3 = locks.lockAll("1", 2, 3);
locks.unlockAll(ls3);
Assert.assertThrows(IllegalArgumentException.class, () -> {
locks.lockAll("1", null);
}, e -> {
Assert.assertContains("Lock key can't be null", e.getMessage());
});
Assert.assertThrows(IllegalArgumentException.class, () -> {
locks.lockAll(null, "1");
}, e -> {
Assert.assertContains("Lock key can't be null", e.getMessage());
});
Assert.assertThrows(IllegalArgumentException.class, () -> {
locks.lockAll(Arrays.asList("1", null, 2).toArray());
}, e -> {
Assert.assertContains("Lock key can't be null", e.getMessage());
});
Assert.assertThrows(IllegalArgumentException.class, () -> {
locks.lockAll(Collections.emptyList().toArray());
}, e -> {
Assert.assertContains("Lock keys can't be null or empty",
e.getMessage());
});
Assert.assertThrows(IllegalArgumentException.class, () -> {
locks.lockAll((Object[]) null);
}, e -> {
Assert.assertContains("Lock keys can't be null or empty",
e.getMessage());
});
Assert.assertThrows(IllegalArgumentException.class, () -> {
locks.unlockAll(null);
}, e -> {
Assert.assertContains("Unlock locks can't be null", e.getMessage());
});
}
}

View File

@ -1,106 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.unit.concurrent;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.junit.Test;
import org.apache.hugegraph.concurrent.AtomicLock;
import org.apache.hugegraph.concurrent.KeyLock;
import org.apache.hugegraph.concurrent.LockGroup;
import org.apache.hugegraph.concurrent.RowLock;
import org.apache.hugegraph.testutil.Assert;
import org.apache.hugegraph.unit.BaseUnitTest;
public class LockGroupTest extends BaseUnitTest {
private static final String GROUP = "LockGroupTest-test-group";
private final LockGroup group = new LockGroup(GROUP);
@Test
public void testLock() {
Lock lock = this.group.lock("lock");
Assert.assertTrue(lock instanceof ReentrantLock);
Lock lock1 = this.group.lock("lock");
Assert.assertSame(lock, lock1);
lock1.lock();
try {
// lock again is OK
lock1.lock();
// lock in other threads
runWithThreads(2, () -> {
Assert.assertFalse(lock1.tryLock());
});
lock1.unlock();
} finally {
lock1.unlock();
}
}
@Test
public void testAtomicLock() {
AtomicLock lock = this.group.atomicLock("lock");
Assert.assertNotNull(lock);
AtomicLock lock1 = this.group.atomicLock("lock");
Assert.assertSame(lock, lock1);
Assert.assertEquals("lock", lock1.name());
}
@Test
public void testReadWriteLock() {
ReadWriteLock lock = this.group.readWriteLock("lock");
Assert.assertTrue(lock instanceof ReentrantReadWriteLock);
ReadWriteLock lock1 = this.group.readWriteLock("lock");
Assert.assertSame(lock, lock1);
}
@Test
public void testKeyLock() {
KeyLock lock = this.group.keyLock("lock");
Assert.assertNotNull(lock);
KeyLock lock1 = this.group.keyLock("lock");
Assert.assertSame(lock, lock1);
}
@Test
public void testKeyLockWithSize() {
KeyLock lock = this.group.keyLock("lock", 10);
Assert.assertNotNull(lock);
KeyLock lock1 = this.group.keyLock("lock");
Assert.assertSame(lock, lock1);
}
@Test
public void testRowLock() {
RowLock<?> lock = this.group.rowLock("lock");
Assert.assertNotNull(lock);
RowLock<?> lock1 = this.group.rowLock("lock");
Assert.assertSame(lock, lock1);
}
@Test
public void testName() {
Assert.assertEquals(GROUP, this.group.name());
}
}

View File

@ -1,125 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.unit.concurrent;
import org.junit.After;
import org.junit.Test;
import org.apache.hugegraph.concurrent.LockGroup;
import org.apache.hugegraph.concurrent.LockManager;
import org.apache.hugegraph.testutil.Assert;
import org.apache.hugegraph.unit.BaseUnitTest;
public class LockManagerTest extends BaseUnitTest {
private static final String GROUP = "LockManagerTest-test-group";
private static final String GROUP2 = GROUP + 2;
@After
public void teardown() {
LockManager manager = LockManager.instance();
if (manager.exists(GROUP)) {
manager.destroy(GROUP);
}
if (manager.exists(GROUP2)) {
manager.destroy(GROUP2);
}
}
@Test
public void testCreate() {
LockManager manager = LockManager.instance();
LockGroup lockGroup = manager.create(GROUP);
Assert.assertNotNull(lockGroup);
Assert.assertEquals(GROUP, lockGroup.name());
Assert.assertTrue(manager.exists(GROUP));
Assert.assertFalse(manager.exists(GROUP2));
LockGroup lockGroup2 = manager.create(GROUP2);
Assert.assertNotNull(lockGroup2);
Assert.assertEquals(GROUP2, lockGroup2.name());
Assert.assertTrue(manager.exists(GROUP2));
Assert.assertThrows(RuntimeException.class, () -> {
manager.create(GROUP);
}, e -> {
Assert.assertContains("LockGroup 'LockManagerTest-test-group' " +
"already exists", e.getMessage());
});
}
@Test
public void testGet() {
LockManager manager = LockManager.instance();
LockGroup lockGroup = manager.create(GROUP);
LockGroup lockGroup2 = manager.create(GROUP2);
Assert.assertSame(lockGroup, manager.get(GROUP));
Assert.assertSame(lockGroup2, manager.get(GROUP2));
Assert.assertSame(lockGroup, manager.get(GROUP));
Assert.assertSame(lockGroup2, manager.get(GROUP2));
Assert.assertThrows(RuntimeException.class, () -> {
manager.get("fake-lock-group");
}, e -> {
Assert.assertContains("LockGroup 'fake-lock-group' " +
"does not exists", e.getMessage());
});
}
@Test
public void testDestroy() {
LockManager manager = LockManager.instance();
LockGroup lockGroup = manager.create(GROUP);
LockGroup lockGroup2 = manager.create(GROUP2);
Assert.assertTrue(manager.exists(GROUP));
Assert.assertTrue(manager.exists(GROUP2));
Assert.assertSame(lockGroup, manager.get(GROUP));
Assert.assertSame(lockGroup2, manager.get(GROUP2));
manager.destroy(GROUP);
Assert.assertFalse(manager.exists(GROUP));
Assert.assertTrue(manager.exists(GROUP2));
Assert.assertThrows(RuntimeException.class, () -> {
manager.get(GROUP);
}, e -> {
Assert.assertContains("does not exists", e.getMessage());
});
Assert.assertSame(lockGroup2, manager.get(GROUP2));
manager.destroy(GROUP2);
Assert.assertFalse(manager.exists(GROUP));
Assert.assertFalse(manager.exists(GROUP2));
Assert.assertThrows(RuntimeException.class, () -> {
manager.get(GROUP);
}, e -> {
Assert.assertContains("does not exists", e.getMessage());
});
Assert.assertThrows(RuntimeException.class, () -> {
manager.get(GROUP2);
}, e -> {
Assert.assertContains("does not exists", e.getMessage());
});
}
}

View File

@ -1,93 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/
package org.apache.hugegraph.unit.concurrent;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import org.apache.hugegraph.util.ExecutorUtil;
import org.junit.Assert;
import org.junit.Test;
import org.apache.hugegraph.concurrent.PausableScheduledThreadPool;
public class PausableScheduledThreadPoolTest {
@Test
public void testScheduleWithFixedDelay() throws InterruptedException {
PausableScheduledThreadPool executor =
ExecutorUtil.newPausableScheduledThreadPool("test");
long period = 500L;
AtomicInteger counter = new AtomicInteger(0);
executor.scheduleWithFixedDelay(() -> {
System.out.println("counter: " + counter.incrementAndGet());
}, period, period, TimeUnit.MILLISECONDS);
Thread.sleep((long) (2.1 * period));
Assert.assertEquals(2, counter.get());
// pause
executor.pauseSchedule();
Thread.sleep(period);
Assert.assertEquals(2, counter.get());
// resume
executor.resumeSchedule();
Thread.sleep((long) (0.5 * period));
Assert.assertEquals(3, counter.get());
Thread.sleep((long) (0.6 * period));
Assert.assertEquals(4, counter.get());
// pause again
executor.pauseSchedule();
executor.shutdown();
executor.awaitTermination(3L, TimeUnit.SECONDS);
}
@Test
public void testScheduleWithFixedRate() throws InterruptedException {
PausableScheduledThreadPool executor =
ExecutorUtil.newPausableScheduledThreadPool(2, "test");
long period = 500L;
AtomicInteger counter = new AtomicInteger(0);
executor.scheduleAtFixedRate(() -> {
System.out.println("counter: " + counter.incrementAndGet());
}, period, period, TimeUnit.MILLISECONDS);
Thread.sleep((long) (2.1 * period));
Assert.assertEquals(2, counter.get());
// pause
executor.pauseSchedule();
Thread.sleep(period);
Assert.assertEquals(2, counter.get());
// resume
executor.resumeSchedule();
Thread.sleep((long) (1.1 * period));
Assert.assertEquals(4, counter.get());
// pause again
executor.pauseSchedule();
executor.shutdownNow();
executor.awaitTermination(3L, TimeUnit.SECONDS);
}
}

Some files were not shown because too many files have changed in this diff Show More