容器 | 大数据笔记
所有文章 标签 分类 关于 DOK文档 科研
容器 | 大数据笔记

目录

Kubeflow-v0.6.1部署采坑记录

runzhliu 收录于 大数据和机器学习
   约 3094 字   预计阅读 7 分钟 

概述

安装条件，可以在 Kubeflow requirements 找到，具体贴出来，因为版本不同，以及 K8S 一些自定义配置会对部署造成影响。

/kubeflow-v0.6.1%E9%83%A8%E7%BD%B2%E9%87%87%E5%9D%91%E8%AE%B0%E5%BD%95/image_1dhspu8gd1od31u54cnpgb52uq9.png

P.S.本文部署的是v0.6.0

部署

部署提供了集中场景，分别对应 GCP, AWS, ACK, IBM Cloud 等等，本文主要讲的场景是你已经有一套 K8S 集群了，👇是我的配置。Deploying Kubeflow on Existing Clusters 后面简称 on EC 好了。

/kubeflow-v0.6.1%E9%83%A8%E7%BD%B2%E9%87%87%E5%9D%91%E8%AE%B0%E5%BD%95/image_1dhsqc07q14en1qf76ef1sabve0m.png

官方文档给出的 on EC 有两种配置的方案，为什么这么麻烦呢？因为 Kubeflow 本身组件太多了，社区有一个配置方案，另外有个 Vendor 也有一个配置方案，大家可以先都了解下，本文用的是社区的配置方案，也叫做 Community maintained。

/kubeflow-v0.6.1%E9%83%A8%E7%BD%B2%E9%87%87%E5%9D%91%E8%AE%B0%E5%BD%95/image_1dhsqv1hrr0i1mlkjojq2la2713.png

重点在下面的执行脚本的过程。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Add kfctl to PATH, to make the kfctl binary easier to use.
export PATH=$PATH:"<path to kfctl>"
export KFAPP="<your choice of application directory name>"
# Installs istio by default. Comment out istio components in the config file to skip istio installation. See https://github.com/kubeflow/kubeflow/pull/3663
export CONFIG="https://raw.githubusercontent.com/kubeflow/kubeflow/master/bootstrap/config/kfctl_k8s_istio.yaml"

kfctl init ${KFAPP} --config=${CONFIG} -V
cd ${KFAPP}
kfctl generate all -V
kfctl apply all -V

因为组件太多，配置很容易出问题，所以社区给的是一个统一的配置文件，当然有些组件是插拔式的，不需要的组件可以注释掉。

如果想提前了解都安装了什么组件，建议看看这个 CONFIG 的内容。

/kubeflow-v0.6.1%E9%83%A8%E7%BD%B2%E9%87%87%E5%9D%91%E8%AE%B0%E5%BD%95/image_1dhsrh21t1du01ga9v2p1uba141e1g.png

如果集群不能访问公网的话，可以将这个文件下载到私有集群上。下图有提到在 kfctl generate 过程中，有下载一些资源。

/kubeflow-v0.6.1%E9%83%A8%E7%BD%B2%E9%87%87%E5%9D%91%E8%AE%B0%E5%BD%95/image_1dhsrprj5fva1blr13it1ig79kq1t.png

按照官网的提示，kubectl -n kubeflow get all 查一下安装的所有 K8S 资源类型。

/kubeflow-v0.6.1%E9%83%A8%E7%BD%B2%E9%87%87%E5%9D%91%E8%AE%B0%E5%BD%95/image_1dhtn73cs1qt61t9016o1qib17qc2q.png

最后附上安装和部署的日志，大家可以根据日志来排查部署的问题。

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471

➜  try-kf kfctl init ${KFAPP} --config=${CONFIG} -V
INFO[0000] Downloading https://raw.githubusercontent.com/kubeflow/kubeflow/master/bootstrap/config/kfctl_k8s_istio.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/127834711/app.yaml  filename="v1alpha1/application_types.go:334"
WARN[0000] Overriding KfDef.Spec.Name; old value kubeflow_app; new value myapp  filename="coordinator/coordinator.go:274"
WARN[0000] Repo manifests exists in app.yaml ignoring version provided by --package-manager  filename="coordinator/coordinator.go:515"
INFO[0000] Creating directory /tmp/try-kf/myapp          filename="coordinator/coordinator.go:384"
INFO[0000] Writing KfDef to /tmp/try-kf/myapp/app.yaml   filename="coordinator/coordinator.go:401"
INFO[0000] Writing stripped KfDef to /tmp/try-kf/myapp/app.yaml  filename="v1alpha1/application_types.go:626"
INFO[0000] Synchronize cache                             filename="coordinator/coordinator.go:431"
INFO[0000] Creating directory /tmp/try-kf/myapp/.cache   filename="v1alpha1/application_types.go:409"
INFO[0000] Fetching https://github.com/kubeflow/manifests/archive/master.tar.gz to /tmp/try-kf/myapp/.cache/manifests  filename="v1alpha1/application_types.go:449"
INFO[0007] Fetch succeeded; LocalPath /tmp/try-kf/myapp/.cache/manifests/manifests-master  filename="v1alpha1/application_types.go:475"
INFO[0007] Fetching https://github.com/kubeflow/kubeflow/archive/master.tar.gz to /tmp/try-kf/myapp/.cache/kubeflow  filename="v1alpha1/application_types.go:449"
INFO[0013] Fetch succeeded; LocalPath /tmp/try-kf/myapp/.cache/kubeflow/kubeflow-master  filename="v1alpha1/application_types.go:475"
INFO[0013] Writing stripped KfDef to /tmp/try-kf/myapp/app.yaml  filename="v1alpha1/application_types.go:626"
INFO[0013] Downloading /tmp/try-kf/myapp/app.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/091930508/app.yaml  filename="v1alpha1/application_types.go:334"
➜  try-kf ls
kfctl myapp
➜  try-kf cd myapp
➜  myapp ls
app.yaml
➜  myapp kfctl generate all -V
INFO[0000] Downloading /tmp/try-kf/myapp/app.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/112839703/app.yaml  filename="v1alpha1/application_types.go:334"
INFO[0000] Writing stripped KfDef to /tmp/try-kf/myapp/app.yaml  filename="v1alpha1/application_types.go:626"
INFO[0000] Downloading /tmp/try-kf/myapp/app.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/514253450/app.yaml  filename="v1alpha1/application_types.go:334"
INFO[0000] Initializing kustomize package map            filename="kustomize/kustomize.go:149"
INFO[0000] Component path map: {}                        filename="kustomize/kustomize.go:170"
INFO[0000] Writing stripped KfDef to /tmp/try-kf/myapp/app.yaml  filename="v1alpha1/application_types.go:626"
INFO[0000] Processing application: istio-crds            filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: istio-install         filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: istio                 filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: application-crds      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: application           filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: metacontroller        filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: argo                  filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: centraldashboard      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: bootstrap             filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: webhook               filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: jupyter-web-app       filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: katib-db              filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: katib-manager         filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: katib-controller      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: katib-ui              filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: metadata              filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: metrics-collector     filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: suggestion            filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: notebook-controller   filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: pytorch-job-crds      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: pytorch-operator      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: spartakus             filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: tensorboard           filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: tf-job-operator       filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: api-service           filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: minio                 filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: mysql                 filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: persistent-agent      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: pipelines-runner      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: pipelines-ui          filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: pipelines-viewer      filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: scheduledworkflow     filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: profiles              filename="kustomize/kustomize.go:589"
INFO[0000] Processing application: seldon-core-operator  filename="kustomize/kustomize.go:589"
➜  myapp kfctl apply all -V
INFO[0000] Downloading /tmp/try-kf/myapp/app.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/984513046/app.yaml  filename="v1alpha1/application_types.go:334"
INFO[0000] Writing stripped KfDef to /tmp/try-kf/myapp/app.yaml  filename="v1alpha1/application_types.go:626"
INFO[0000] Downloading /tmp/try-kf/myapp/app.yaml to /var/folders/0g/qn4zfbj15m73ccbhz16_b_f00000gn/T/028543101/app.yaml  filename="v1alpha1/application_types.go:334"
INFO[0000] Initializing a default restConfig for Kubernetes  filename="kustomize/kustomize.go:248"
INFO[0000] namespace: kubeflow                           filename="kustomize/kustomize.go:265"
INFO[0000] Creating namespace: kubeflow                  filename="kustomize/kustomize.go:268"
INFO[0000] creating CustomResourceDefinition/adapters.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/apikeys.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/attributemanifests.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/authorizations.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/bypasses.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/certificates.certmanager.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/challenges.certmanager.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/checknothings.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/circonuses.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/cloudwatches.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/clusterissuers.certmanager.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/clusterrbacconfigs.rbac.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/deniers.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/destinationrules.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/dogstatsds.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/edges.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/envoyfilters.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/fluentds.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/gateways.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/handlers.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/httpapispecbindings.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/httpapispecs.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/instances.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/issuers.certmanager.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/kubernetesenvs.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/kuberneteses.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/listcheckers.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/listentries.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/logentries.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/memquotas.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/meshpolicies.authentication.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/metrics.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/noops.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/opas.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/orders.certmanager.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/policies.authentication.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/prometheuses.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/quotas.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/quotaspecbindings.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/quotaspecs.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/rbacconfigs.rbac.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/rbacs.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/redisquotas.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/reportnothings.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/rules.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/serviceentries.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/servicerolebindings.rbac.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/serviceroles.rbac.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/sidecars.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/signalfxs.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/solarwindses.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/stackdrivers.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/statsds.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/stdios.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/templates.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/tracespans.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/virtualservices.networking.istio.io  filename="kustomize/kustomize.go:430"
INFO[0000] creating CustomResourceDefinition/zipkins.config.istio.io  filename="kustomize/kustomize.go:430"
INFO[0001] creating Namespace/istio-system               filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-citadel-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-cleanup-secrets-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-egressgateway-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-galley-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-grafana-post-install-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-ingressgateway-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-mixer-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-multi           filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-pilot-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-security-post-install-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/istio-sidecar-injector-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/kiali-service-account  filename="kustomize/kustomize.go:430"
INFO[0001] creating ServiceAccount/prometheus            filename="kustomize/kustomize.go:430"
INFO[0001] creating Role/istio-ingressgateway-sds        filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-citadel-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-cleanup-secrets-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-egressgateway-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-galley-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-grafana-post-install-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-ingressgateway-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-mixer-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-pilot-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-reader             filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-sidecar-injector-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/kiali-viewer             filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/kiali                    filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/prometheus-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRole/istio-security-post-install-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating RoleBinding/istio-ingressgateway-sds  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-citadel-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-cleanup-secrets-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-egressgateway-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-galley-admin-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-grafana-post-install-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-ingressgateway-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-kiali-admin-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-mixer-admin-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-multi       filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-pilot-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-sidecar-injector-admin-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/prometheus-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ClusterRoleBinding/istio-security-post-install-role-binding-istio-system  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-galley-configuration  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-galley-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-istio-mesh-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-istio-performance-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-istio-service-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-istio-workload-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-mixer-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-configuration-dashboards-pilot-dashboard  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana-custom-resources  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-grafana              filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-security-custom-resources  filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio-sidecar-injector     filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/istio                      filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/kiali                      filename="kustomize/kustomize.go:430"
INFO[0001] creating ConfigMap/prometheus                 filename="kustomize/kustomize.go:430"
INFO[0001] creating Secret/kiali                         filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/grafana                      filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-citadel                filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-egressgateway          filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-galley                 filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-ingressgateway         filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-pilot                  filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-policy                 filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-sidecar-injector       filename="kustomize/kustomize.go:430"
INFO[0001] creating Service/istio-telemetry              filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/jaeger-agent                 filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/jaeger-collector             filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/jaeger-query                 filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/kiali                        filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/prometheus                   filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/tracing                      filename="kustomize/kustomize.go:430"
INFO[0002] creating Service/zipkin                       filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/grafana                   filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-citadel             filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-egressgateway       filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-galley              filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-ingressgateway      filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-pilot               filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-policy              filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-sidecar-injector    filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-telemetry           filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/istio-tracing             filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/kiali                     filename="kustomize/kustomize.go:430"
INFO[0002] creating Deployment/prometheus                filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-egressgateway  filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-galley     filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-ingressgateway  filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-pilot      filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-policy     filename="kustomize/kustomize.go:430"
INFO[0002] creating PodDisruptionBudget/istio-telemetry  filename="kustomize/kustomize.go:430"
INFO[0002] creating MutatingWebhookConfiguration/istio-sidecar-injector  filename="kustomize/kustomize.go:430"
INFO[0002] creating HorizontalPodAutoscaler/istio-egressgateway  filename="kustomize/kustomize.go:430"
INFO[0002] creating HorizontalPodAutoscaler/istio-ingressgateway  filename="kustomize/kustomize.go:430"
INFO[0002] creating HorizontalPodAutoscaler/istio-pilot  filename="kustomize/kustomize.go:430"
INFO[0002] creating HorizontalPodAutoscaler/istio-policy  filename="kustomize/kustomize.go:430"
INFO[0002] creating HorizontalPodAutoscaler/istio-telemetry  filename="kustomize/kustomize.go:430"
INFO[0002] creating Job/istio-cleanup-secrets-1.1.6      filename="kustomize/kustomize.go:430"
INFO[0002] creating Job/istio-grafana-post-install-1.1.6  filename="kustomize/kustomize.go:430"
INFO[0002] creating Job/istio-security-post-install-1.1.6  filename="kustomize/kustomize.go:430"
INFO[0002] creating attributemanifest/istioproxy         filename="kustomize/kustomize.go:430"
INFO[0002] creating attributemanifest/kubernetes         filename="kustomize/kustomize.go:430"
INFO[0002] creating handler/kubernetesenv                filename="kustomize/kustomize.go:430"
INFO[0002] creating handler/prometheus                   filename="kustomize/kustomize.go:430"
INFO[0002] creating handler/stdio                        filename="kustomize/kustomize.go:430"
INFO[0002] creating kubernetes/attributes                filename="kustomize/kustomize.go:430"
INFO[0002] creating logentry/accesslog                   filename="kustomize/kustomize.go:430"
INFO[0002] creating logentry/tcpaccesslog                filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/requestcount                  filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/requestduration               filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/requestsize                   filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/responsesize                  filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/tcpbytereceived               filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/tcpbytesent                   filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/tcpconnectionsclosed          filename="kustomize/kustomize.go:430"
INFO[0002] creating metric/tcpconnectionsopened          filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/kubeattrgenrulerule             filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/promhttp                        filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/promtcpconnectionclosed         filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/promtcpconnectionopen           filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/promtcp                         filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/stdiotcp                        filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/stdio                           filename="kustomize/kustomize.go:430"
INFO[0002] creating rule/tcpkubeattrgenrulerule          filename="kustomize/kustomize.go:430"
INFO[0003] creating DestinationRule/istio-policy         filename="kustomize/kustomize.go:430"
INFO[0003] creating DestinationRule/istio-telemetry      filename="kustomize/kustomize.go:430"
INFO[0003] creating ConfigMap/istio-parameters-9m7m4fhdb8  filename="kustomize/kustomize.go:430"
INFO[0003] creating Gateway/kubeflow-gateway             filename="kustomize/kustomize.go:430"
INFO[0003] creating ServiceEntry/google-api-entry        filename="kustomize/kustomize.go:430"
INFO[0003] creating ServiceEntry/google-storage-api-entry  filename="kustomize/kustomize.go:430"
INFO[0003] creating VirtualService/google-api-vs         filename="kustomize/kustomize.go:430"
INFO[0003] creating VirtualService/google-storage-api-vs  filename="kustomize/kustomize.go:430"
INFO[0003] creating VirtualService/grafana-vs            filename="kustomize/kustomize.go:430"
INFO[0003] creating ClusterRbacConfig/default            filename="kustomize/kustomize.go:430"
INFO[0003] creating CustomResourceDefinition/applications.app.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0003] creating ServiceAccount/application-controller-service-account  filename="kustomize/kustomize.go:430"
INFO[0003] creating ClusterRole/application-controller-cluster-role  filename="kustomize/kustomize.go:430"
INFO[0003] creating ClusterRoleBinding/application-controller-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0003] creating ConfigMap/application-controller-parameters  filename="kustomize/kustomize.go:430"
INFO[0003] creating Service/application-controller-service  filename="kustomize/kustomize.go:430"
INFO[0003] creating StatefulSet/application-controller-stateful-set  filename="kustomize/kustomize.go:430"
INFO[0003] creating Application/kubeflow                 filename="kustomize/kustomize.go:430"
INFO[0004] creating CustomResourceDefinition/compositecontrollers.metacontroller.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0004] creating CustomResourceDefinition/controllerrevisions.metacontroller.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0004] creating CustomResourceDefinition/decoratorcontrollers.metacontroller.k8s.io  filename="kustomize/kustomize.go:430"
INFO[0004] creating ServiceAccount/meta-controller-service  filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRoleBinding/meta-controller-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0004] creating StatefulSet/metacontroller           filename="kustomize/kustomize.go:430"
INFO[0004] creating CustomResourceDefinition/workflows.argoproj.io  filename="kustomize/kustomize.go:430"
INFO[0004] creating ServiceAccount/argo-ui               filename="kustomize/kustomize.go:430"
INFO[0004] creating ServiceAccount/argo                  filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRole/argo-ui                  filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRole/argo                     filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRoleBinding/argo-ui           filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRoleBinding/argo              filename="kustomize/kustomize.go:430"
INFO[0004] creating ConfigMap/workflow-controller-configmap  filename="kustomize/kustomize.go:430"
INFO[0004] creating ConfigMap/workflow-controller-parameters  filename="kustomize/kustomize.go:430"
INFO[0004] creating Service/argo-ui                      filename="kustomize/kustomize.go:430"
INFO[0004] creating Deployment/argo-ui                   filename="kustomize/kustomize.go:430"
INFO[0004] creating Deployment/workflow-controller       filename="kustomize/kustomize.go:430"
INFO[0004] creating VirtualService/argo-ui               filename="kustomize/kustomize.go:430"
INFO[0004] creating ServiceAccount/centraldashboard      filename="kustomize/kustomize.go:430"
INFO[0004] creating Role/centraldashboard                filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRole/centraldashboard         filename="kustomize/kustomize.go:430"
INFO[0004] creating RoleBinding/centraldashboard         filename="kustomize/kustomize.go:430"
INFO[0004] creating ClusterRoleBinding/centraldashboard  filename="kustomize/kustomize.go:430"
INFO[0004] creating ConfigMap/parameters                 filename="kustomize/kustomize.go:430"
INFO[0004] creating Service/centraldashboard             filename="kustomize/kustomize.go:430"
INFO[0004] creating Deployment/centraldashboard          filename="kustomize/kustomize.go:430"
INFO[0004] creating VirtualService/centraldashboard      filename="kustomize/kustomize.go:430"
INFO[0005] creating ServiceAccount/admission-webhook-bootstrap-service-account  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRole/admission-webhook-bootstrap-cluster-role  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRoleBinding/admission-webhook-bootstrap-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0005] creating ConfigMap/admission-webhook-bootstrap-config-map  filename="kustomize/kustomize.go:430"
INFO[0005] creating StatefulSet/admission-webhook-bootstrap-stateful-set  filename="kustomize/kustomize.go:430"
INFO[0005] creating CustomResourceDefinition/poddefaults.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0005] creating ServiceAccount/admission-webhook-service-account  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRole/admission-webhook-cluster-role  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRoleBinding/admission-webhook-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0005] creating ConfigMap/admission-webhook-admission-webhook-parameters  filename="kustomize/kustomize.go:430"
INFO[0005] creating Service/admission-webhook-service    filename="kustomize/kustomize.go:430"
INFO[0005] creating Deployment/admission-webhook-deployment  filename="kustomize/kustomize.go:430"
INFO[0005] creating MutatingWebhookConfiguration/admission-webhook-mutating-webhook-configuration  filename="kustomize/kustomize.go:430"
INFO[0005] creating ServiceAccount/jupyter-web-app-service-account  filename="kustomize/kustomize.go:430"
INFO[0005] creating Role/jupyter-web-app-jupyter-notebook-role  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRole/jupyter-web-app-cluster-role  filename="kustomize/kustomize.go:430"
INFO[0005] creating RoleBinding/jupyter-web-app-jupyter-notebook-role-binding  filename="kustomize/kustomize.go:430"
INFO[0005] creating ClusterRoleBinding/jupyter-web-app-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0005] creating ConfigMap/jupyter-web-app-config     filename="kustomize/kustomize.go:430"
INFO[0005] creating ConfigMap/jupyter-web-app-parameters  filename="kustomize/kustomize.go:430"
INFO[0005] creating Service/jupyter-web-app-service      filename="kustomize/kustomize.go:430"
INFO[0005] creating Deployment/jupyter-web-app-deployment  filename="kustomize/kustomize.go:430"
INFO[0006] creating Application/jupyter-web-app          filename="kustomize/kustomize.go:430"
INFO[0006] creating VirtualService/jupyter-web-app       filename="kustomize/kustomize.go:430"
INFO[0006] creating Secret/katib-db-secrets              filename="kustomize/kustomize.go:430"
INFO[0006] creating Service/katib-db                     filename="kustomize/kustomize.go:430"
INFO[0006] creating Deployment/katib-db                  filename="kustomize/kustomize.go:430"
INFO[0006] creating PersistentVolumeClaim/katib-mysql    filename="kustomize/kustomize.go:430"
INFO[0006] creating Service/katib-manager-rest           filename="kustomize/kustomize.go:430"
INFO[0006] creating Service/katib-manager                filename="kustomize/kustomize.go:430"
INFO[0006] creating Deployment/katib-manager-rest        filename="kustomize/kustomize.go:430"
INFO[0006] creating Deployment/katib-manager             filename="kustomize/kustomize.go:430"
INFO[0007] creating CustomResourceDefinition/experiments.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0007] creating CustomResourceDefinition/trials.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0007] creating ServiceAccount/katib-controller      filename="kustomize/kustomize.go:430"
INFO[0007] creating ClusterRole/katib-controller         filename="kustomize/kustomize.go:430"
INFO[0007] creating ClusterRoleBinding/katib-controller  filename="kustomize/kustomize.go:430"
INFO[0007] creating ConfigMap/trial-template             filename="kustomize/kustomize.go:430"
INFO[0007] creating Secret/katib-controller              filename="kustomize/kustomize.go:430"
INFO[0007] creating Service/katib-controller             filename="kustomize/kustomize.go:430"
INFO[0007] creating Deployment/katib-controller          filename="kustomize/kustomize.go:430"
INFO[0007] creating ServiceAccount/katib-ui              filename="kustomize/kustomize.go:430"
INFO[0007] creating ClusterRole/katib-ui                 filename="kustomize/kustomize.go:430"
INFO[0007] creating ClusterRoleBinding/katib-ui          filename="kustomize/kustomize.go:430"
INFO[0007] creating ConfigMap/katib-parameters           filename="kustomize/kustomize.go:430"
INFO[0007] creating Service/katib-ui                     filename="kustomize/kustomize.go:430"
INFO[0007] creating Deployment/katib-ui                  filename="kustomize/kustomize.go:430"
INFO[0007] creating VirtualService/katib-ui              filename="kustomize/kustomize.go:430"
INFO[0007] creating ServiceAccount/metadata-ui           filename="kustomize/kustomize.go:430"
INFO[0007] creating Role/metadata-ui                     filename="kustomize/kustomize.go:430"
INFO[0007] creating RoleBinding/metadata-ui              filename="kustomize/kustomize.go:430"
INFO[0007] creating ConfigMap/metadata-ui-parameters-b6c8ghff7c  filename="kustomize/kustomize.go:430"
INFO[0007] creating Secret/metadata-db-secrets           filename="kustomize/kustomize.go:430"
INFO[0007] creating Service/metadata-db                  filename="kustomize/kustomize.go:430"
INFO[0007] creating Service/metadata-service             filename="kustomize/kustomize.go:430"
INFO[0007] creating Service/metadata-ui                  filename="kustomize/kustomize.go:430"
INFO[0007] creating Deployment/metadata-ui               filename="kustomize/kustomize.go:430"
INFO[0007] creating Deployment/metadata-db               filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/metadata-deployment       filename="kustomize/kustomize.go:430"
INFO[0008] creating VirtualService/metadata-ui           filename="kustomize/kustomize.go:430"
INFO[0008] creating PersistentVolumeClaim/metadata-mysql  filename="kustomize/kustomize.go:430"
INFO[0008] creating ServiceAccount/metrics-collector     filename="kustomize/kustomize.go:430"
INFO[0008] creating ClusterRole/metrics-collector        filename="kustomize/kustomize.go:430"
INFO[0008] creating ClusterRoleBinding/metrics-collector  filename="kustomize/kustomize.go:430"
INFO[0008] creating ConfigMap/metrics-collector-template  filename="kustomize/kustomize.go:430"
INFO[0008] creating Service/katib-suggestion-bayesianoptimization  filename="kustomize/kustomize.go:430"
INFO[0008] creating Service/katib-suggestion-grid        filename="kustomize/kustomize.go:430"
INFO[0008] creating Service/katib-suggestion-hyperband   filename="kustomize/kustomize.go:430"
INFO[0008] creating Service/katib-suggestion-nasrl       filename="kustomize/kustomize.go:430"
INFO[0008] creating Service/katib-suggestion-random      filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/katib-suggestion-bayesianoptimization  filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/katib-suggestion-grid     filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/katib-suggestion-hyperband  filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/katib-suggestion-nasrl    filename="kustomize/kustomize.go:430"
INFO[0008] creating Deployment/katib-suggestion-random   filename="kustomize/kustomize.go:430"
INFO[0009] creating CustomResourceDefinition/notebooks.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0009] creating ServiceAccount/notebook-controller-service-account  filename="kustomize/kustomize.go:430"
INFO[0009] creating ClusterRole/notebook-controller-role  filename="kustomize/kustomize.go:430"
INFO[0009] creating ClusterRoleBinding/notebook-controller-role-binding  filename="kustomize/kustomize.go:430"
INFO[0009] creating ConfigMap/notebook-controller-parameters  filename="kustomize/kustomize.go:430"
INFO[0009] creating Service/notebook-controller-service  filename="kustomize/kustomize.go:430"
INFO[0009] creating Deployment/notebook-controller-deployment  filename="kustomize/kustomize.go:430"
INFO[0009] creating Application/notebook-controller      filename="kustomize/kustomize.go:430"
INFO[0009] creating CustomResourceDefinition/pytorchjobs.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0010] creating ServiceAccount/pytorch-operator      filename="kustomize/kustomize.go:430"
INFO[0010] creating ClusterRole/pytorch-operator         filename="kustomize/kustomize.go:430"
INFO[0010] creating ClusterRoleBinding/pytorch-operator  filename="kustomize/kustomize.go:430"
INFO[0010] creating ConfigMap/pytorch-operator-config    filename="kustomize/kustomize.go:430"
INFO[0010] creating Service/pytorch-operator             filename="kustomize/kustomize.go:430"
INFO[0010] creating Deployment/pytorch-operator          filename="kustomize/kustomize.go:430"
INFO[0011] creating ServiceAccount/spartakus             filename="kustomize/kustomize.go:430"
INFO[0011] creating ClusterRole/spartakus                filename="kustomize/kustomize.go:430"
INFO[0011] creating ClusterRoleBinding/spartakus         filename="kustomize/kustomize.go:430"
INFO[0011] creating ConfigMap/spartakus-parameters       filename="kustomize/kustomize.go:430"
INFO[0011] creating Deployment/spartakus-volunteer       filename="kustomize/kustomize.go:430"
INFO[0011] creating ConfigMap/parameters-dgd4h256h5      filename="kustomize/kustomize.go:430"
INFO[0011] creating Service/tensorboard                  filename="kustomize/kustomize.go:430"
INFO[0011] creating Deployment/tensorboard               filename="kustomize/kustomize.go:430"
INFO[0011] creating VirtualService/tensorboard           filename="kustomize/kustomize.go:430"
INFO[0012] creating CustomResourceDefinition/tfjobs.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0012] creating ServiceAccount/tf-job-dashboard      filename="kustomize/kustomize.go:430"
INFO[0012] creating ServiceAccount/tf-job-operator       filename="kustomize/kustomize.go:430"
INFO[0012] creating ClusterRole/tf-job-dashboard         filename="kustomize/kustomize.go:430"
INFO[0012] creating ClusterRole/tf-job-operator          filename="kustomize/kustomize.go:430"
INFO[0012] creating ClusterRoleBinding/tf-job-dashboard  filename="kustomize/kustomize.go:430"
INFO[0012] creating ClusterRoleBinding/tf-job-operator   filename="kustomize/kustomize.go:430"
INFO[0012] creating ConfigMap/parameters-dgd4h256h5      filename="kustomize/kustomize.go:430"
INFO[0012] creating ConfigMap/tf-job-operator-config     filename="kustomize/kustomize.go:430"
INFO[0012] creating Service/tf-job-dashboard             filename="kustomize/kustomize.go:430"
INFO[0012] creating Service/tf-job-operator              filename="kustomize/kustomize.go:430"
INFO[0012] creating Deployment/tf-job-dashboard          filename="kustomize/kustomize.go:430"
INFO[0012] creating Deployment/tf-job-operator           filename="kustomize/kustomize.go:430"
INFO[0012] creating VirtualService/tf-job-dashboard      filename="kustomize/kustomize.go:430"
INFO[0013] creating ServiceAccount/ml-pipeline           filename="kustomize/kustomize.go:430"
INFO[0013] creating Role/ml-pipeline                     filename="kustomize/kustomize.go:430"
INFO[0013] creating RoleBinding/ml-pipeline              filename="kustomize/kustomize.go:430"
INFO[0013] creating Service/ml-pipeline                  filename="kustomize/kustomize.go:430"
INFO[0013] creating Deployment/ml-pipeline               filename="kustomize/kustomize.go:430"
INFO[0013] creating ConfigMap/pipeline-minio-parameters  filename="kustomize/kustomize.go:430"
INFO[0013] creating Secret/mlpipeline-minio-artifact     filename="kustomize/kustomize.go:430"
INFO[0013] creating Service/minio-service                filename="kustomize/kustomize.go:430"
INFO[0014] creating Deployment/minio                     filename="kustomize/kustomize.go:430"
INFO[0014] creating PersistentVolumeClaim/minio-pv-claim  filename="kustomize/kustomize.go:430"
INFO[0014] creating ConfigMap/pipeline-mysql-parameters  filename="kustomize/kustomize.go:430"
INFO[0014] creating Service/mysql                        filename="kustomize/kustomize.go:430"
INFO[0014] creating Deployment/mysql                     filename="kustomize/kustomize.go:430"
INFO[0015] creating PersistentVolumeClaim/mysql-pv-claim  filename="kustomize/kustomize.go:430"
INFO[0015] creating ServiceAccount/ml-pipeline-persistenceagent  filename="kustomize/kustomize.go:430"
INFO[0015] creating ClusterRole/ml-pipeline-persistenceagent  filename="kustomize/kustomize.go:430"
INFO[0015] creating ClusterRoleBinding/ml-pipeline-persistenceagent  filename="kustomize/kustomize.go:430"
INFO[0015] creating Deployment/ml-pipeline-persistenceagent  filename="kustomize/kustomize.go:430"
INFO[0016] creating ServiceAccount/pipeline-runner       filename="kustomize/kustomize.go:430"
INFO[0016] creating ClusterRole/pipeline-runner          filename="kustomize/kustomize.go:430"
INFO[0016] creating ClusterRoleBinding/pipeline-runner   filename="kustomize/kustomize.go:430"
INFO[0017] creating ServiceAccount/ml-pipeline-ui        filename="kustomize/kustomize.go:430"
INFO[0017] creating Role/ml-pipeline-ui                  filename="kustomize/kustomize.go:430"
INFO[0017] creating RoleBinding/ml-pipeline-ui           filename="kustomize/kustomize.go:430"
INFO[0017] creating ConfigMap/ui-parameters-hb792fcf5d   filename="kustomize/kustomize.go:430"
INFO[0017] creating Service/ml-pipeline-tensorboard-ui   filename="kustomize/kustomize.go:430"
INFO[0017] creating Service/ml-pipeline-ui               filename="kustomize/kustomize.go:430"
INFO[0017] creating Deployment/ml-pipeline-ui            filename="kustomize/kustomize.go:430"
INFO[0017] creating VirtualService/ml-pipeline-tensorboard-ui  filename="kustomize/kustomize.go:430"
INFO[0017] creating VirtualService/ml-pipeline-ui        filename="kustomize/kustomize.go:430"
INFO[0017] creating CustomResourceDefinition/viewers.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0017] creating ServiceAccount/ml-pipeline-viewer-crd-service-account  filename="kustomize/kustomize.go:430"
INFO[0017] creating ClusterRole/ml-pipeline-viewer-controller-role  filename="kustomize/kustomize.go:430"
INFO[0017] creating ClusterRoleBinding/ml-pipeline-viewer-crd-role-binding  filename="kustomize/kustomize.go:430"
INFO[0018] creating Deployment/ml-pipeline-viewer-controller-deployment  filename="kustomize/kustomize.go:430"
E0810 20:09:32.999888   31811 memcache.go:135] couldn't get resource list for kubeflow.org/v1beta1: the server could not find the requested resource
INFO[0018] creating CustomResourceDefinition/scheduledworkflows.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0018] creating ServiceAccount/ml-pipeline-scheduledworkflow  filename="kustomize/kustomize.go:430"
INFO[0018] creating Role/ml-pipeline-scheduledworkflow   filename="kustomize/kustomize.go:430"
INFO[0018] creating ClusterRoleBinding/ml-pipeline-scheduledworkflow  filename="kustomize/kustomize.go:430"
INFO[0019] creating Deployment/ml-pipeline-scheduledworkflow  filename="kustomize/kustomize.go:430"
E0810 20:09:34.026474   31811 memcache.go:135] couldn't get resource list for kubeflow.org/v1beta1: the server could not find the requested resource
INFO[0019] creating CustomResourceDefinition/profiles.kubeflow.org  filename="kustomize/kustomize.go:430"
INFO[0019] creating ServiceAccount/profiles-controller-service-account  filename="kustomize/kustomize.go:430"
INFO[0019] creating ServiceAccount/profiles-default-service-account  filename="kustomize/kustomize.go:430"
INFO[0019] creating Role/profiles-default-role           filename="kustomize/kustomize.go:430"
INFO[0019] creating RoleBinding/profiles-default-role-binding  filename="kustomize/kustomize.go:430"
INFO[0019] creating ClusterRoleBinding/profiles-cluster-role-binding  filename="kustomize/kustomize.go:430"
INFO[0019] creating ConfigMap/profiles-profiles-parameters-t4hd69826h  filename="kustomize/kustomize.go:430"
INFO[0020] creating Service/profiles-kfam                filename="kustomize/kustomize.go:430"
INFO[0020] creating Deployment/profiles-deployment       filename="kustomize/kustomize.go:430"
INFO[0020] creating VirtualService/kfam                  filename="kustomize/kustomize.go:430"
INFO[0021] creating CustomResourceDefinition/seldondeployments.machinelearning.seldon.io  filename="kustomize/kustomize.go:430"
INFO[0021] creating ServiceAccount/seldon-manager        filename="kustomize/kustomize.go:430"
INFO[0021] creating ClusterRole/seldon-operator-manager-role  filename="kustomize/kustomize.go:430"
INFO[0021] creating ClusterRoleBinding/seldon-operator-manager-rolebinding  filename="kustomize/kustomize.go:430"
INFO[0021] creating Secret/seldon-operator-webhook-server-secret  filename="kustomize/kustomize.go:430"
INFO[0021] creating Service/seldon-operator-controller-manager-service  filename="kustomize/kustomize.go:430"
INFO[0021] creating StatefulSet/seldon-operator-controller-manager  filename="kustomize/kustomize.go:430"
INFO[0021] creating Application/seldon-core-operator     filename="kustomize/kustomize.go:430"

总结

Kubeflow 的部署虽然可以通过脚本一键部署，但是里面涉及很多模块，也涉及很多 K8S 的知识，所以需要对这些都有一定的了解，才容易部署成功。Kubeflow 还是很多值得探索的地方的，所以后面还会有更多的分析文章。

警告
本文最后更新于 2017年2月1日，文中内容可能已过时，请谨慎参考。
更新于 2017-02-01
 机器学习, Kubeflow
返回 | 主页