Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PR for Action Rekognition feature monitoring #221

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .coveralls.yml

This file was deleted.

161 changes: 161 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
name: CI

# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ main ,dev]
pull_request:
branches: [ dev,main ]
jobs:
docker-job:
strategy:
matrix:
os: [ubuntu-20.04,ubuntu-18.04]
version: [3.6]
runs-on: ${{ matrix.os }}

# service containers to run with `postgres-job`
services:
# label used to access the service container
postgres:
# Docker Hub image
image: postgres:latest
# service environment variables
# `POSTGRES_HOST` is `postgres`
env:
# optional (defaults to `postgres`)
POSTGRES_DB: postgres_db
# required
POSTGRES_PASSWORD: postgres
# optional (defaults to `5432`)
POSTGRES_PORT: 5432
# optional (defaults to `postgres`)
POSTGRES_USER: postgres_db
ports:
# maps tcp port 5432 on service container to the host
- 5432:5432
# set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:


#Installing PostgressSQL client

- name: Install PostgreSQL client
run: |
sudo apt-get update
sudo apt-get install --yes postgresql-client
echo postgree installed
# Setting Up Postgres Database

- name: Setup Database database
run: |
echo creating database
psql -h localhost -U postgres_db -c "CREATE DATABASE pmr"
echo database created
psql -h localhost -U postgres_db -c "CREATE USER admin WITH PASSWORD 'admin'"
psql -h localhost -U postgres_db -c "ALTER ROLE admin SET client_encoding TO 'utf8'"
psql -h localhost -U postgres_db -c "ALTER ROLE admin SET default_transaction_isolation TO 'read committed'"
psql -h localhost -U postgres_db -c "ALTER ROLE admin SET timezone TO 'UTC'"
psql -h localhost -U postgres_db -c "ALTER USER admin CREATEDB"
psql -h localhost -U postgres_db -c "ALTER DATABASE pmr OWNER TO admin"
echo "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | sudo tee /etc/apt/sources.list.d/tensorflow-serving.list && \
curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | sudo apt-key add -
sudo apt-get update
sudo apt-get install tensorflow-model-server
echo postgres sttted up

env:
# Setting up POSRGRES Enviorment
POSTGRES_HOST: localhost
POSTGRES_USER: postgres_db
POSTGRES_PASSWORD: postgres

POSTGRES_PORT: 5432
PGPASSWORD: postgres


- uses: actions/checkout@v3
- uses: actions/setup-python@v3.1.2
with:
python-version: 3.6

# Dependency Installation

- name: Install Dependencies
run: |
echo python version =
python --version
sudo apt-get install python3-dev
pip3 install --upgrade pip
pip3 install -r requirements.txt
pip3 install tensorflow-serving-api==2.5.2
pip3 install pytest
echo installed dependencies
flake8 --ignore=E501,F821,E265,E741 .
# Downloading Test Files

- name: Download Files
run: |
mkdir tests/testdata
cd tests/testdata
wget https://www.dropbox.com/s/1bnxg32zvgjv0pl/compareImage.jpeg
wget https://www.dropbox.com/s/1bnxg32zvgjv0pl/compareImage.jpeg
wget https://www.dropbox.com/s/x3qpga9gc4ifamn/t1.png
wget https://www.dropbox.com/s/l5t09lp8u4ok593/t2.jpeg
wget https://www.dropbox.com/s/hzlpo74tk0xwzzh/t3.jpeg
wget https://www.dropbox.com/s/lni50cgunua5mij/test1.mp4
wget https://www.dropbox.com/s/wm3llx0ydbnq8mn/test2.mp4
wget https://www.dropbox.com/s/ato4fie6k3lmctu/test3.mp4
wget https://www.dropbox.com/s/ifd7254x29oxjze/test4.mp4
wget https://www.dropbox.com/s/iwtgwz24eipd629/obj1.mp4
wget https://www.dropbox.com/s/ull2tqlou1p8l16/obj2.mp4
wget https://www.dropbox.com/s/3w5ghr5jj6opr58/scene1.mp4
wget https://www.dropbox.com/s/ij5hj4hznczvfcw/text.mp4
cd ../..
cd media
mkdir object
mkdir nsfw
cd ..
cd corelib/model
mkdir facenet
cd facenet
wget https://www.dropbox.com/s/jm8grrifh5yk7is/2017.zip?dl=1 -O 2017.zip
unzip 2017.zip
rm 2017.zip
cd ..
mkdir tfs
cd tfs
wget --load-cookies /tmp/cookies.txt "https://drive.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://drive.google.com/uc?export=download&id=12yE9v8dWeVidqxseUXidaDoS_VZpVOp1' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=12yE9v8dWeVidqxseUXidaDoS_VZpVOp1" -O module.zip && rm -rf /tmp/cookies.txt
unzip module.zip
rm module.zip
cd ../../..
cd data
mkdir text_reco
cd text_reco
wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=1zmxO5NoEqTdYRGHXalaK7bn2m27egQMN' -O char_dict_en.json
wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=1XGO2ycQcBYtwxT_KanapuoR6LeGR4s_E' -O ord_map_en.json
cd ../..
sudo apt update
sudo apt install ffmpeg
export DJANGO_SETTINGS_MODULE="Rekognition.settings"
- run: |
export DJANGO_SETTINGS_MODULE="Rekognition.settings"
set -x
docker version
docker run --rm hello-world
docker pull tensorflow/serving:nightly-devel
echo $(pwd)
docker run -d -t -p 8500:8500 -p 8501:8501 -v /home/runner/work/Rekognition/Rekognition/corelib/model/tfs/model_volume:/home/ tensorflow/serving --model_config_file=/home/configs/models.conf
echo aagye
python manage.py flush --no-input
python manage.py migrate
python manage.py runserver 8000 &
sleep 10
python -m pytest --import-mode=append /home/runner/work/Rekognition/Rekognition/tests/ --no-header -vv
echo done all
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,5 @@ __pycache__
env/
db.sqlite3
*.log
myenv/
myen/
66 changes: 0 additions & 66 deletions .travis.yml

This file was deleted.

1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ Google Summer Of Code Project under CCExtractor Development
[![GPLv3 license](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://github.com/ccextractor/Rekognition/blob/master/LICENSE)

---
![bannerimage](https://github.com/CCExtractor/Rekognition/banner.png)
This project aims at providing a free alternative to Amazon Rekognition services.


Expand Down
75 changes: 64 additions & 11 deletions contributing.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,33 @@
Table of contents:
- [Contributing to Rekognition](#Contributing-to-Rekognition)
- [Making a PR](#making-a-pr)
- [Asking for help](#asking-for-help)
- [Development environment setup](#Development-environment-setup)

As beginners, navigating the codebase and finding your way out of the documentation can become difficult. This page will help you understand everything about contributing to howdoi and the best practices in open source as well.

## Contributing to Rekognition
- Follow the page Setting up the development environment for setting up the development environment for Rekognition.
- Finding your first issue
- Go to issues in the Rekognition repo.
- Find the issues which you might be interested to work on. Or, you can also come up with your own ideas of improving the code.
- After finding the issue you are interested in : If the issue is an existing one, comment on the issue and ask for it to be assigned to you. Or, if the issue is unlisted and new , create a new issue and fill every information needed in the issues template provided by howdoi and ask for it to be assigned to you.
- After receiving confirmation, start working on the issue and whenever and wherever help is needed, comment on the issue itself describing your query in detail.
- A good guide on how to collaborate efficiently can be found here.

## Making a PR
- After you have worked on the issue and fixed it, we need to merge it from your forked repository into the Rekognition repository by making a PR.
- Each PR made should pass all the tests. We have new Github Actions in place for CI/CD.
- Once your commit passes all the tests, make a PR and wait for it to be reviewed and merged.


## Asking for help
- At times, help is needed while solving the issue. We recommend the following step for asking for help when you get stuck:
- Read from our documentation to see if your question has already been answered.
- Comment on the issue you are working on describing in detail what problems you are facing.
- Make sure to write your query in detail and if it is a bug, include steps to reproduce it.
- If you are not working on any issue and have a question to be answered, open a new issue on Github and wait for a reply.

# Development Environment setup
## Install python 3.6
```
Expand All @@ -6,14 +36,29 @@ sudo apt-get update
sudo apt-get install python3.6
```

## For MacOS:
```
brew install python
```


## Clone the repository and setup venv
```
git clone https://github.com/pymit/Rekognition
git clone https://github.com/CCExtractor/Rekognition
cd Rekognition
./setup.sh
../setup.sh
source myenv/bin/activate
```

### For MacOS:
git clone https://github.com/CCExtractor/Rekognition
./setup.sh
cd Rekognition
python3 -m virtualenv myenv
source $PWD/myenv/bin/activate
pip install -r ../requirements.txt

NOTE: Sometimes an error "permission denied" may be shown when you try to run `setup.sh`. For this, try: `chmod 755 setup.sh` in root directory to change permissions.
***
## Postgres setup

Expand All @@ -30,16 +75,20 @@ source myenv/bin/activate
ALTER USER admin CREATEDB;
ALTER DATABASE pmr OWNER TO admin;
***
## Postgres setup for MacOS

## ReactJS setup for frontend

git clone https://github.com/pymit/RekoUI
cd RekoUI
sudo apt install npm
sudo npm install -g npm@latest
npm install
npm start
***
brew update
brew install postgresql
brew services start postgresql
psql postgres
CREATE DATABASE pmr;
CREATE USER admin WITH PASSWORD 'admin';
ALTER ROLE admin SET client_encoding TO 'utf8';
ALTER ROLE admin SET default_transaction_isolation TO 'read committed';
ALTER ROLE admin SET timezone TO 'UTC';
ALTER USER admin CREATEDB;
ALTER DATABASE pmr OWNER TO admin;
***

## Downloading the models
##### current directory Rekognition
Expand Down Expand Up @@ -107,3 +156,7 @@ python manage.py runserver 8000
Django app can be accessed at http://localhost:8000

ReactJS app can be accessed at http://localhost:3000




Loading