Commit Graph

2147 Commits

Author SHA1 Message Date
cheyang 00741af240
Build docker images for 0.9.3 (#3762)
Signed-off-by: cheyang <cheyang@163.com>
2024-03-13 21:19:50 +08:00
TzZtzt e0184cff87
Merge pull request from GHSA-wx8q-4gm9-rj2g
* Fix JuicefsRuntime: escape customized string before constructing commands

add escapeBashStr

Signed-off-by: xixi <hexilee@juicedata.io>

avoid bash -c in operations

Signed-off-by: xixi <hexilee@juicedata.io>

fix GetUsedSpace and GetFileCount

Signed-off-by: xixi <hexilee@juicedata.io>

move EscapeBashStr to pkg/utils/security

Signed-off-by: xixi <hexilee@juicedata.io>

add left

Signed-off-by: xixi <hexilee@juicedata.io>

resume GetFileCount

Signed-off-by: xixi <hexilee@juicedata.io>

Escape value.Configs.Name

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

Fix unit tests

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

Upgrade juicefs helm chart version to 0.2.16

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Fix JuicefsRuntime: escape customized string before constructing commands

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
Co-authored-by: xixi <hexilee@juicedata.io>
2024-03-13 20:38:50 +08:00
Twilight 86371688ef
fix rbac role issue (#3753)
Signed-off-by: chenqiming <whqscqm@outlook.com>
2024-03-07 13:15:06 +00:00
TzZtzt 212d6e811d
del metadata sync && del duplicate metrics (#3380) (#3738)
* del metadata sync && del duplicate metrics



* log level



---------

Signed-off-by: zwwhdls <zww@hdls.me>
Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
Co-authored-by: Weiwei <zww@hdls.me>
2024-02-28 08:12:56 +00:00
cheyang e43fb0c4f8
Remove fsgroup from alluxio runtime (#3635) (#3654)
* Remove fsgroup from alluxio runtime, To #53506158



* Remove fsgroup from alluxio runtime, To #53506158



* Remove fsgroup from alluxio runtime, To #53506158



---------

Signed-off-by: cheyang <cheyang@163.com>
2023-12-21 15:56:10 +08:00
cheyang 1f98a657cd
enhance: remove jindoruntime's fsGroup (#3632) (#3641)
* enhance: remove jindoruntime's fsGroup (#3632)

* Remove jindofsx's securityContext.fsGroup

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
Signed-off-by: cheyang <cheyang@163.com>

* Mount ufs volumes according to datasets' accessModes, To #53506158

Signed-off-by: cheyang <cheyang@163.com>

* Mount ufs volumes according to datasets' accessModes, To #53506158

Signed-off-by: cheyang <cheyang@163.com>

---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
Signed-off-by: cheyang <cheyang@163.com>
Co-authored-by: TzZtzt <trafalgarz@outlook.com>
2023-12-15 02:14:55 +00:00
cheyang 69ca087107
Prepare 0.9.3, To #53506158 (#3638)
Signed-off-by: cheyang <cheyang@163.com>
2023-12-12 16:48:06 +08:00
cheyang 880474b992
Build docker images for release 0.9.2, To #51793350 (#3466)
Signed-off-by: cheyang <cheyang@163.com>
2023-09-19 06:02:06 +00:00
cheyang 02f70acc7e
Fix the problem of using option to append Dataset public option (#3454) (#3459)
* Fix the problem of using option to append Dataset public option (#3454)

* Fix the problem of using option to append Dataset public option

Signed-off-by: lgy1027 <lgy10271416@gmail.com>

* Add Unit Test for alluxio engine genUFSMountOptions

Signed-off-by: ZhangXiaozheng <zhang__xiaozheng@outlook.com>

---------

Signed-off-by: lgy1027 <lgy10271416@gmail.com>
Signed-off-by: ZhangXiaozheng <zhang__xiaozheng@outlook.com>
Co-authored-by: liguoyu3564 <liguoyu@leinao.ai>
Co-authored-by: ZhangXiaozheng <zhang__xiaozheng@outlook.com>
Signed-off-by: cheyang <cheyang@163.com>

* Fix api issue for genUFSMountOptions, To #51793350

Signed-off-by: cheyang <cheyang@163.com>

---------

Signed-off-by: lgy1027 <lgy10271416@gmail.com>
Signed-off-by: ZhangXiaozheng <zhang__xiaozheng@outlook.com>
Signed-off-by: cheyang <cheyang@163.com>
Co-authored-by: li_max <lgy10271416@gmail.com>
Co-authored-by: liguoyu3564 <liguoyu@leinao.ai>
Co-authored-by: ZhangXiaozheng <zhang__xiaozheng@outlook.com>
2023-09-18 15:44:26 +08:00
TzZtzt 7b01a4c59f
Cherry-pick to 0.9: fix csi plugin concurrency issue on FuseRecovery and NodeUnpublishVolume (#3448) (#3453)
* Bugfix: ignore not connected error in NodeUnpublishVolume (#3445)

* ignore not connected error in NodeUnpublishVolume

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* fix check nil error

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* simplify error judgment

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

---------

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* bugfix: fix csi plugin concurrency issue on FuseRecovery and NodeUnpublishVolume (#3448)

* Add comments for NodeUnpublishVolume

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Refactor NodeUnpublishVolume code

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* FuseRecovery uses volume locks to avoid race conditions

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Refactor node server with codes.Internal error code

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Rename CSI Config to RunningContext

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Fix github actions checks

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Fix lock release

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Refactor recover logic

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>
Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
Co-authored-by: wangshulin <89928606+wangshli@users.noreply.github.com>
2023-09-13 06:05:59 +00:00
TzZtzt 6acbbcbb75
umountDuplicate larger than the threshold (#3429) (#3452)
* Bugfix: remove umountDuplicate and add warning event (#3403)

* Bugfix: remove umountDuplicate and add warning event

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* fix ut error

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* using GetIntValueFromEnv and set default recover warning threshold as 50

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* switch for eventReason and add log

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

---------

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* umountDuplicate larger than the threshold (#3429)

* umountDuplicate larger than the threshold

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* update warning log

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* add dataset event record

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* update eventReason and message

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

---------

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>
Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>
Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
Co-authored-by: wangshulin <89928606+wangshli@users.noreply.github.com>
2023-09-13 03:56:59 +00:00
TzZtzt 2a141d783c
fail fast with wrong kubelet rootdir (#3331) (#3451)
* fail fast with wrong kubelet rootdir



* check /pods dir



* check kubelet rootdir /plugins



* function check_directory



* add more info



---------

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>
Co-authored-by: wangshulin <89928606+wangshli@users.noreply.github.com>
2023-09-13 03:27:59 +00:00
TzZtzt 76e2621a32
check path existence and lock in NodeUnpublishVolume (#3284) (#3450)
* check path existence and lock in NodeUnpublishVolume



* use path level lock and lock for NodePublishVolume



* small fix



* return codes.aborted when the lock is not acquired



* check target path validity



* adjust order of check path and acquire lock



---------

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>
Co-authored-by: wangshulin <89928606+wangshli@users.noreply.github.com>
2023-09-13 03:05:58 +00:00
cheyang 987e6e7605
enhancement: remove juicefs fuse readiness probe (#3435) (#3436)
* Remove useless readiness probe on JuiceFS fuse



* Remove useless readiness probe on JuiceFS fuse



---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
Signed-off-by: cheyang <cheyang@163.com>
Co-authored-by: TzZtzt <trafalgarz@outlook.com>
2023-09-05 14:14:44 +08:00
cheyang 3ca580bf22
Prepare 0.9.2, To #50714625 (#3374)
Signed-off-by: cheyang <cheyang@163.com>
2023-08-04 20:51:47 +08:00
cheyang b5ee92a1d3
Build docker image for release 0.9.1, To #50179395 (#3294)
Signed-off-by: cheyang <cheyang@163.com>
2023-06-15 11:29:30 +00:00
cheyang 36c75158bc
bugfix: fix csi plugin loop mount bug (#3287) (#3290)
* Fix CSI node plugin loop mount error



---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
Signed-off-by: cheyang <cheyang@163.com>
Co-authored-by: TzZtzt <trafalgarz@outlook.com>
2023-06-14 07:02:29 +00:00
cheyang e79e93bc9c
Fix default Alluxio writetype config (#3262) (#3265)
Signed-off-by: ZhangXiaozheng <zhang__xiaozheng@outlook.com>
Signed-off-by: cheyang <cheyang@163.com>
Co-authored-by: Xiaozheng Zhang <63225111+zhang-x-z@users.noreply.github.com>
2023-06-04 10:29:27 +08:00
cheyang 8c2023403e
Prepare 0.9.1, To #48332603 (#3263)
Signed-off-by: cheyang <cheyang@163.com>
2023-06-02 17:43:08 +08:00
cheyang 53c240c631
Build docker image for 0.9.0 (#3253)
* Build docker image for 0.9.0, To #48332556

Signed-off-by: cheyang <cheyang@163.com>

* Build docker image for 0.9.0, To #48332556

Signed-off-by: cheyang <cheyang@163.com>

---------

Signed-off-by: cheyang <cheyang@163.com>
2023-05-26 09:50:30 +00:00
TzZtzt f78f11cd76
bugfix: fix jindo-fuse postStart script to make sure fuse mountpoint is ready (#3252)
* Fix jindo-fuse postStart script

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Fix jindo test case

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
2023-05-26 05:42:28 +00:00
cheyang 33e247700d
Build docker image for 0.9.0, To #48332556 (#3250)
Signed-off-by: cheyang <cheyang@163.com>
2023-05-24 14:18:29 +00:00
TzZtzt f6945d03de
bugfix: fix post start script failure when launching jindo fuse sidecar (#3243)
* Fix jindo fuse sidecar mountpoint checking

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Fix unit tests

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Add e2e testcase for JindoRuntime fuse sidecar mode

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Fix e2e testcases

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
2023-05-24 21:12:42 +08:00
cheyang 39fb3dbc6d
Prepare release 0.9 (#3249)
* Prepare release 0.9, To #48332556

Signed-off-by: cheyang <cheyang@163.com>

* Prepare release 0.9, To #48332556

Signed-off-by: cheyang <cheyang@163.com>

* Prepare release 0.9, To #48332556

Signed-off-by: cheyang <cheyang@163.com>

---------

Signed-off-by: cheyang <cheyang@163.com>
2023-05-24 12:07:27 +00:00
cheyang 4c782561d8
Build docker images for metrics port, To #49004965 (#3244)
Signed-off-by: cheyang <cheyang@163.com>
2023-05-24 06:52:27 +00:00
cheyang 5008f1fdbd
Update architecture, To #48327952 (#3247)
Signed-off-by: cheyang <cheyang@163.com>
2023-05-24 10:17:14 +08:00
TzZtzt 37e17c63cc
optimize: optimize makefile targets and speed up docker build (#3246)
* Remove dlv binary in Dockerfiles

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Optimize makefile targets

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Fix csi fuse recovery testcase

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Move "OpenAPI validation check" to lint CI task

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
2023-05-24 10:16:54 +08:00
frankleaf 0f7d87c0b6
fix trun off metrics port with jindofuse (#3241)
Signed-off-by: frankleaf <frank.wt@alibaba-inc.com>
2023-05-21 11:21:31 +08:00
TzZtzt 3ef8adc9cf
bugfix: Set Dataset.Spec.Mounts as a required property (#3240)
* Set Dataset.Spec.Mounts as a required property

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Generate openapi

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
2023-05-19 09:58:06 +00:00
dependabot[bot] 8322397d02
Bump github.com/spf13/cobra from 1.6.1 to 1.7.0 (#3234)
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.6.1...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-18 01:58:04 +00:00
cheyang fab0b3f7c3
Support libfuse3 for alluxio 2.9, To #48332603 (#3236)
Signed-off-by: cheyang <cheyang@163.com>
2023-05-17 05:46:03 +00:00
Xiaozheng Zhang 66d64fdef8
Adapt to libfuse 3 (#3235)
* Adapt to libfuse 3

Signed-off-by: ZhangXiaozheng <zhang__xiaozheng@outlook.com>

* Fix unit test

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: ZhangXiaozheng <zhang__xiaozheng@outlook.com>
Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
Co-authored-by: trafalgarzzz <trafalgarz@outlook.com>
2023-05-16 13:07:03 +00:00
cheyang 886a5c114c
Integrate with KServe (#3202)
* Integrate with KServe, To #48853387

Signed-off-by: cheyang <cheyang@163.com>

* Remove update from operations in kserve, To #48853387

Signed-off-by: cheyang <cheyang@163.com>

---------

Signed-off-by: cheyang <cheyang@163.com>
2023-05-15 06:38:02 +00:00
cheyang d103d62f8f
Build docker images for jindoruntime secretmount enhancement, To #48853401 (#3233)
Signed-off-by: cheyang <cheyang@163.com>
Co-authored-by: TzZtzt <trafalgarz@outlook.com>
2023-05-15 02:32:01 +00:00
cheyang 5661f14105
Build docker images for removing secret in jindoruntime, To #48327952 (#3232)
Signed-off-by: cheyang <cheyang@163.com>
2023-05-15 02:10:00 +00:00
dependabot[bot] d751eb1003
Bump github.com/prometheus/client_golang from 1.14.0 to 1.15.1 (#3230)
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.14.0 to 1.15.1.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.14.0...v1.15.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-13 07:10:59 +00:00
frankleaf 51518dc7ed
Fix JindoRuntime support ak with secret (#3218)
* fix support ak from file

Signed-off-by: frankleaf <frank.wt@alibaba-inc.com>

* fix changelog

Signed-off-by: frankleaf <frank.wt@alibaba-inc.com>

* fix judge version

Signed-off-by: frankleaf <frank.wt@alibaba-inc.com>

* fix test

Signed-off-by: frankleaf <frank.wt@alibaba-inc.com>

* fix test

Signed-off-by: frankleaf <frank.wt@alibaba-inc.com>

* fix test

Signed-off-by: frankleaf <frank.wt@alibaba-inc.com>

---------

Signed-off-by: frankleaf <frank.wt@alibaba-inc.com>
2023-05-12 10:23:58 +00:00
yangjun289519474 8671ff8a0d
fix(efcruntime): add prefix efc to /dev/shm volume when eci (#3229)
Signed-off-by: zhanyi <hanson.yj@alibaba-inc.com>
2023-05-12 09:54:58 +00:00
dependabot[bot] f4e4a3122e
Bump golang.org/x/net from 0.8.0 to 0.10.0 (#3222)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.8.0 to 0.10.0.
- [Commits](https://github.com/golang/net/compare/v0.8.0...v0.10.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-12 02:14:58 +00:00
wangshulin e53470dcac
update helm to 3.11.3 (#3214)
Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>
2023-05-12 02:14:05 +00:00
wangshulin 3f76277815
add e2e test to validate dataset using pvc subpath (#3039)
* add e2e test to validate dataset using pvc subpath

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

using NodeSelector to pin local pv/pvc in current node

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

using WebUFS Dataset as base PVC

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* add subpath in alluxio and goosefs charts

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

* set dataset shared

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>

---------

Signed-off-by: wangshulin <wangshulin@smail.nju.edu.cn>
2023-05-10 11:39:58 +00:00
cheyang a5b1e331a0
Build docker images for default reinvocationPolicy as IfNeeded, To #48327952 (#3225)
Signed-off-by: cheyang <cheyang@163.com>
2023-05-10 11:17:57 +00:00
yangjun289519474 c2bee26c49
fix(efcruntime): set Privileged=true when init-fuse start (#3221)
Signed-off-by: zhanyi <hanson.yj@alibaba-inc.com>
2023-05-09 04:25:55 +00:00
Lize Cai 247e3042d8
set default reinvocationPolicy to IfNeeded (#3219)
Signed-off-by: Lize Cai <lize.cai@sap.com>
2023-05-09 03:00:55 +00:00
TzZtzt 0fe1bd9b77
fix(efcruntime): Add runtime identity info and fix readiness probe for EFCRuntime (#3217)
* Add runtime identity info to EFC

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Fix EFC readiness probe

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
2023-05-08 11:43:54 +00:00
yangjun289519474 a618c0be54
fix(efcruntime): remove init container and sessmgr in efc-master yaml (#3216)
Signed-off-by: zhanyi <hanson.yj@alibaba-inc.com>
2023-05-08 11:42:54 +00:00
cheyang b22700c4e0
Build docker images for fixing rbacs and limit CSI Plugin's node related access, To #48327952 (#3213)
Signed-off-by: cheyang <cheyang@163.com>
2023-05-08 09:10:53 +00:00
TzZtzt 77c8110a3d
Merge pull request from GHSA-93xx-cvmc-9w3v
* Fix rbacs and limit CSI Plugin's node related access

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

* Update change log

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>

---------

Signed-off-by: trafalgarzzz <trafalgarz@outlook.com>
2023-05-08 14:46:24 +08:00
cheyang 1cc14b31b9
Update docker images for efcruntime on serverless container, To #48853387 (#3208)
Signed-off-by: cheyang <cheyang@163.com>
2023-05-06 14:00:45 +00:00
dependabot[bot] 085b23e2fd
Bump github.com/go-logr/logr from 1.2.3 to 1.2.4 (#2893)
Bumps [github.com/go-logr/logr](https://github.com/go-logr/logr) from 1.2.3 to 1.2.4.
- [Release notes](https://github.com/go-logr/logr/releases)
- [Changelog](https://github.com/go-logr/logr/blob/master/CHANGELOG.md)
- [Commits](https://github.com/go-logr/logr/compare/v1.2.3...v1.2.4)

---
updated-dependencies:
- dependency-name: github.com/go-logr/logr
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-06 06:23:45 +00:00