Some details need more research, but works for now!

MuJoCo is a advanced physical simulation tool, developed by Emo Todorov for Roboti LLC
mujoco-py is a glue to let people use MuJoCo in python, developed by OpenAI Robotics team
OpenAI Gym is a modular python package to provide environments for RL, developed by OpenAI

Install MuJoCo

Get License

1. Download mjpro150_linux.zip from https://www.roboti.us/index.html
2. Get 1 year “Personal License” or 30 days “Trial License”
3. Download mjkey.txt that will be emailed

Install MuJoCo

1. Make directory in root named /.mujoco (in root for convenience in integration with mujoco-py)
2. Unzip mjpro150_linux.zip to /.mujoco which will make /.mujoco/mjpro150/
3. Save mjkey.txt to /.mujoco and /.mujoco/mjpro150/bin/ (one in /.mujoco/mjpro150/bin/ is for ./simulate to test MuJoCo working)
4. Add environment variables to ~/.bashrc

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/.mujoco/mjpro150/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib/nvidia-384"

Test MuJoCo

1. Go to /.mujoco/mjpro150/bin
2. Run ./simulate

Install mujoco-py

1. Clone mujoco-py from https://github.com/openai/mujoco-py
2. Install to python3 (using sudo python3 -m pip install -e . –no-cache)
3. Add environment variables to ~/.bashrc

export MUJOCO_PY_MJPRO_PATH="/.mujoco/mjpro150"
export MUJOCO_PY_MUJOCO_PATH="/.mujoco"
export MUJOCO_PY_MJKEY_PATH="/.mujoco/mjkey.txt"

Install OpenAI Gym

Recommend installing this after MuJoCo and mujoco-py.

1. Install to python3 (using sudo python3 -m pip install gym[ALL])

Test All Together

1. Test using python script (if this works, you are good to go)

import gym

env = gym.make('Humanoid-v2')
env.reset()
for _ in range(1000):
    env.render()
    action = env.action_space.sample()
    env.step(action)

Reference

https://deeprobotics.wordpress.com/2018/01/23/installing-mujoco-and-integrating-it-with-python-on-ubuntu/
https://github.com/openai/mujoco-py/issues/190