cdl-k8s-core 0.1.4

Connected Data Lake Framework
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

# Connected Data Lake

## Usage

```bash
docker run --rm 'quay.io/ulagbulag/connected-data-lake:latest'
```

## Install K8S Operator

```bash
# Register the Connected Data Lake repository
helm repo add cdl "https://smartx-team.github.io/connected-data-lake"

# Deploy a Connected Data Lake operator
helm install -n "cdl-operator" "cdl-operator" "cdl/cdl-operator"
```

### Install Dependencies on Ubuntu 24.04

```bash
# Install os dependencies
sudo apt-get update
sudo apt-get install -y \
  build-essential \
  fuse \
  libfuse-dev \
  libprotoc-dev \
  protobuf-compiler \
  rustup

# Install & Update the latest stable rust
rustup default stable
```

## Build on the local machine

### Build Requirements

- gcc
- fuse
- protobuf
- rust >=1.82

### Build CDL rust CLI

```bash
cargo run --release --
```

### Build CDL python API

Please check your python virtual environment (i.e. conda) before running.

```bash
cd python
maturin develop --release
```

### Build K8S Operator

Please check your kubernetes config file `~/.kube/config` before running.

```bash
cargo run --package 'cdl-k8s-operator' --release --
```