DevOps
- Compile all tools
- Compile python: Build the python extention
- Pack all tools to external environment (Please Clone MR_Scripts git repository and see script under Bash-scripts):
In the remote machine, unzip the mes_full.tar with "tar -xvf mes_full.tar" and run "medial_earlysign_setup.sh" to setup environment. The environment is similar to what we have in the docker: Ubuntu docker (medial_dev). We can also install a docker if possible. In my opinion docker is 2nd priority. This is based on those actions. If you don't want to to do all of them:
| Command | Description | output path | Where to extract |
|---|---|---|---|
full_build.x86-64.sh |
Builds all our tools for X86_64 computers: Flow, bootstrap_app, etc. |
/nas1/UsersData/git/MR/Tools/AllTools/Linux/Release/bin_apps.x86-64.tar.bz2 | #copy to file to /earlysign/bins cd /earlysign/bins tar -xvf bin_apps.x86-64.tar.bz2 |
pack_etl.sh |
Packs all ETL Infra code | /server/Linux/${USER%-*}/ETL.tar.bz2. USER might be: git if run from "full_pack_all.sh" or your username without "-internal" |
#copy to file to /earlysign/scripts cd /earlysign/scripts tar -xvf ETL.tar.bz2 |
| pack_libs.sh | Packs all "External libs" needed for "full_build.x86-64.sh". For example, xgboost, lightGBM & boost. There are not suppose to be "updates" in here. |
/server/Linux/${USER%-*}/libs.tar.bz2. USER might be: git if run from "full_pack_all.sh" or your username without "-internal" |
#copy to file to /earlysign/libs cd /earlysign/libs tar -xvf libs.tar.bz2 |
| pack_scripts.sh | Packs all our helper scripts like: AutoValidation kits, bootstrap_format.py, plot.pt, paste.pl, etc. | /server/Linux/${USER%-*}/scripts.tar.bz2 USER might be: git if run from "full_pack_all.sh" or your username without "-internal" |
#copy to file to /earlysign/scripts cd /earlysign/scripts tar -xvf scripts.tar.bz2 |
| build_py_wrapper.sh | Builds the python wrapper for our infrastructure. | /server/Linux/${USER%-*}/PY.tar.bz2 USER might be: git if run from "full_pack_all.sh" or your username without "-internal" |
#copy to file to /earlysign/libs cd /earlysign/libs tar -xvf PY.tar.bz2 |
| Additional helper scripts: | A setup script that setup/update the environment: /nas1/UsersData/git/MR/Projects/Scripts/Bash-Scripts/usefull: medial_earlysign_setup.sh, medial_earlysign_resetup.sh |
/nas1/UsersData/${USER%-*}/MR/Projects/Scripts/Bash-Scripts/usefull USER might be: git if run from "full_pack_all.sh" or your username without "-internal" |
- Jupyter is down in the internal?