# Setting up the environment

## General overview

Most library dependencies can be installed from PyPI using pip. There are different target requirements files.
Using a dedicated virtualenv is recommended. To install everything for a target run
`pip install -r requirements/<target>.<platform>.txt` or `pip-sync requirements/<target>.<platform>.txt`. The
latter will remove not needed packages.
These will only work if all packages are available for the current platform specifically. For those that have to be
installed from source packages, all build dependencies - like C compiler - must be set up, and the development
libraries must be available globally. Additionally, the `clarabot-crypto` must be available and the `clarabot-testing`
must be available for development.

Running Nano from the source or packaging it will fetch the Solr and Tika dependencies automatically. When those are
updated (or downloaded for the first time) the jdeps tool generates their java dependencies for jlink to compile a
minimal JRE for running and distribution. This takes several minutes.

System requirements:
    - Python 3.9
    - wxWidgets 3+ (compatible with wxPython 4.1)
    - protobuf library
    - dbus library (linux)

### Windows 10+

Python is compiled with MSVC for Windows. We need to install it (and the Windows 10 SDK) for some binary extensions
that are not distributed in pre-built packages. For details see https://wiki.python.org/moin/WindowsCompilers .
Installing the integrated cmake as well is recommended as we'll need it.

**gcld3** needs Google's protobuf library to be built. The protobuf-compiler is available pre-built, but we can't use
it, since we also need the library bindings. Follow the general instructions for building the dependency available at
https://github.com/protocolbuffers/protobuf/blob/master/cmake/README.md . Options to use: x64 arch, build from the
latest stable git checkout (3.15.6 at the time of writing), release build, use dynamic linking (we have to for python),
use cmake to create nmake configuration, make, check, install. (zlib is not needed)
Copy the `bin`, `include` and `lib` to the virtualenv: `Scripts`, `include` and `libs`. The upstream package's build
script is broken on Windows. Install from this fork:
`pip install git+https://github.com/Traktormaster/cld3@fix-windows#egg=gcld3`
After installation the `libprotobuf.dll` has to be copied next to the pyd extension to be automatically loaded at
runtime. It is at `<venv>\lib\site-packages\gcld3`. See https://docs.python.org/3/library/os.html#os.add_dll_directory
for more information.

## Setup for packaging

1. Firstly clone the branch of `clarabot-nano` with which you want to work.
   Check in the `requirements/base.<platform>.txt` for the required version of `clarabot-crypto`.
   
2. Clone each of the dependent repositories, checkout the required version's tag and build their source-distribution package. Example:
   ```
   git clone git@dc.clarabot.zrt:Clarabot/clarabot-crypto.git
   cd clarabot-crypto
   git checkout v0.8.1
   python setup.py sdist
   ```
   Package should be in `dist` folder.

3. Create a virtualenv to work in `python -m venv <name>` and activate it `source <name>/bin/activate`.
   
   Install some utilities, then the custom packages, then sync the environment:
   ```
   pip install wheel
   pip install pip-tools
   pip install clarabot-crypto-xxx.tar.gz
   pip-sync clarabot-nano/requirements/pkg.<platform>.txt
   ```

4. Build the frozen app and the package.

   In the `clarabot-nano` repository. (add it to `PYTHONPATH` envvar if necessary)

   ```
   python -m dev_tools.build
   ```
   
    NOTE: If you see `Exception: Site cleared, try again...` just try again...

## Building for Linux on CentOS7

Install CentOS 7 with `GNOME Desktop` base environment, select `Development Tools`

1. Add EPEL and install dependencies
   ```
   sudo yum install epel-release
   sudo yum repolist && sudo yum update
   sudo yum install libffi libffi-devel openssl-libs openssl-devel sqlite sqlite-devel readline readline-devel libuuid libuuid-devel uuid uuid-devel protobuf protobuf-devel wxGTK wxGTK-devel wxGTK3 wxGTK3-devel patchelf java-11-openjdk java-11-openjdk-devel java-11-openjdk-jmods
   ```

2. Select `java-11-openjdk` as default Java version
   ```
   sudo alternatives --config java
   ```

3. Build python
   - Download and extract desired source tar(https://github.com/python/cpython/tags)
      ```
      wget https://github.com/python/cpython/archive/refs/tags/v3.9.10.tar.gz
      tar -xf v3.9.10.tar.gz
      cd cpython-3.9.10
      ```

   - Configure and compile
      ```
      mkdir ~/python-build
      export CFLAGS="-march=core2 -mtune=corei7" CXXFLAGS="-march=core2 -mtune=corei7"
      ./configure --enable-shared --prefix=${HOME}/python-build
      make -j${nproc} && make install
      ```

   - Set LD path
      ```
      export LD_LIBRARY_PATH=${HOME}/python-build/lib
      ```
4. Clone and checkout compatible clarabot-nano and clarabot-crypto repos

5. Build clarabot-crypto sdist
   ```
   ~/python-build/bin/python3 setup.py sdist
   ```
   In case pip is not installed for some reason:
   ```
   ~/python-build/bin/python3 -m ensurepip --upgrade
   ```

6. Create venv and install dependencies
   ```
   ~/python-build/bin/python3 -m venv ~/build-venv
   source ~/build-venv/bin/activate
   pip install wheel
   pip install pip-tools
   pip install clarabot-crypto/dist/clarabot-crypto-x.x.x.tar.gz
   CXXFLAGS="--std=c++11" CFLAGS="--std=c++11" pip-sync clarabot-nano/requirements/pkg.linux.txt 
   ```

7. Build clarabot-nano
   ```
   cd clarabot-nano
   python -m dev_tools.build
   ```