Thursday, 17 December 2020

Webcam capture with ffmpeg and OpenCV from Jupyter Notebook

I want to share here my experience with using OpenCV and ffmpeg to capture a webcam output.


Setup:
  • Jupyter notebook running in jupyter-lab
  • Ubuntu 20.04
  • USB web camera
Goal:
  • Capture and display frames from the webcam

OpenCV: Video I/O with OpenCV Overview says that OpenCV: cv::VideoCapture Class calls video I/O backends (APIs) depending on which one is available.

To find out what backends (VideoCaptureAPIs) are available we can use the following code:

import cv2

# cv2a.videoio_registry.getBackends() returns list of all available backends.
availableBackends = [cv2.videoio_registry.getBackendName(b) for b in cv2.videoio_registry.getBackends()]
print(availableBackends)

# Returns list of available backends which works via cv::VideoCapture(int index)
availableCameraBackends = [cv2.videoio_registry.getBackendName(b) for b in cv2.videoio_registry.getCameraBackends()]
print(availableBackends)

The output in my case was: 

['FFMPEG', 'GSTREAMER', 'CV_IMAGES', 'CV_MJPEG']
['FFMPEG', 'GSTREAMER', 'CV_IMAGES', 'CV_MJPEG']

Let's see what is each of these backends:

• FFMPEG is a multimedia framework which can record, convert and stream audio and video.

It contains libavcodec, libavutil, libavformat, libavfilter, libavdevice, libswscale and libswresample which can be used by applications. As well as ffmpeg, ffplay and ffprobe which can be used by end users for transcoding and playing.

• GSTREAMER is a pipeline-based multimedia framework with similar capabilities as ffmpeg.

• CV_IMAGES -  OpenCV Image Sequence (e.g. img_%02d.jpg). Matches cv2.CAP_IMAGES API ID.

• CV_MJPEG - Built-in OpenCV MotionJPEG codec (used for reading video files). Matches cv2.CAP_OPENCV_MJPEG video capture API.

I was surprised to see GSTREAMER listed above as VideoCaptureAPIs documentation says

Backends are available only if they have been built with your OpenCV binaries. 

...and OpenCV package installed in my environment was built only with FFMPEG support:

>>> import cv2
>>> cv2.getBuildInformation()
...
Video I/O:\n    DC1394:                      NO\n    FFMPEG:                      YES\n      avcodec:                   YES (58.35.100)\n      avformat:                  YES (58.20.100)\n      avutil:                    YES (56.22.100)\n      swscale:                   YES (5.3.100)\n      avresample:                YES (4.0.0)\n\n  
...

...which can also be verifed by looking the cmake config in the repository (opencv-feedstock/build.sh at master · conda-forge/opencv-feedstock):

-DWITH_FFMPEG=1     \
-DWITH_GSTREAMER=0  \

Although my conda environment contained all relevant packages:

(my-env) $ conda list | grep 'opencv\|ffmpeg\|gstreamer'
ffmpeg                    4.1.3                h167e202_0    conda-forge
gstreamer                 1.14.5               h36ae1b5_2    conda-forge
opencv                    4.1.0            py36h79d2e43_1    conda-forge

...it is important to know that having ffmpeg and gstreamer packages installed means only that we have their binaries installed (executables and .so libraries) but not Python bindings (modules) or their OpenCV plugins. We are able to launch these applications from terminal but can't import them in Python code.

I tried to force using FFMPEG:

import cv2

deviceId = "/dev/video0"

# videoCaptureApi = cv2.CAP_ANY       # autodetect default API
videoCaptureApi = cv2.CAP_FFMPEG
# videoCaptureApi = cv2.CAP_GSTREAMER 
cap = cv2.VideoCapture("/dev/video2", videoCaptureApi)

cap = cv2.VideoCapture(deviceId)
cap.open(deviceId)
if not cap.isOpened():
    raise RuntimeError("ERROR! Unable to open camera")

try:
    while True:
        ret, frame = cap.read()
        cv2.imshow('frame', frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
finally:        
    cap.release()
    cv2.destroyAllWindows()

...but cell execution would fail with:

RuntimeError: ERROR! Unable to open camera

I checked ($ v4l2-ctl --list-devices) - my webcam was indeed with index 2. As this was failing at the very beginning I decided to open python interpreter console and debug there only the isolated code snippet which opens the camera:

(my-env) $ export OPENCV_LOG_LEVEL=DEBUG; export OPENCV_VIDEOIO_DEBUG=1

(my-env) $ python 
Python 3.6.6 | packaged by conda-forge | (default, Oct 12 2018, 14:43:46) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cap = cv2.VideoCapture("/dev/video2", cv2.CAP_FFMPEG)
[ WARN:0] VIDEOIO(FFMPEG): trying capture filename='/dev/video2' ...
[ WARN:0] VIDEOIO(FFMPEG): can't create capture

I also tried to force using Gstreamer to no avail (which was expected):

>>> cap = cv2.VideoCapture("/dev/video2", cv2.CAP_GSTREAMER)
[ WARN:0] VIDEOIO(GSTREAMER): trying capture filename='/dev/video2' ...
[ INFO:0] VideoIO pluigin (GSTREAMER): glob is 'libopencv_videoio_gstreamer*.so', 1 location(s)
[ INFO:0]     - /home/bojan/anaconda3/envs/my-env/lib/python3.6/site-packages/../..: 0
[ INFO:0] Found 0 plugin(s) for GSTREAMER
[ WARN:0] VIDEOIO(GSTREAMER): backend is not available (plugin is missing, or can't be loaded due dependencies or it is not compatible)

Indeed ~/anaconda3/envs/my-env/lib did not contain ffmpeg plugin (libopencv_videoio_ffmpeg*.so files) or Gstreamer plugin (libopencv_videoio_gstreamer*.so files).

These plugins are installed only if OpenCV is build with following CMake options:

- DWITH_FFMPEG=1     \
-DVIDEOIO_PLUGIN_LIST=ffmpeg

...or (for Gstreamer):

-DWITH_GSTREAMER=1 \
-DVIDEOIO_PLUGIN_LIST=gstreamer \

...and apart from WITH_FFMPEG no other were used in the cmake config that was used to build OpenCV package installed in my environment.

As I didn't want to compile OpenCV myself but to achieve my goal with what I have I decided to see if I can run ffmpg process to stream camera output into a pipe and then read the binary information from it and convert it into frames:

import os
import tempfile
import subprocess
import cv2
import numpy as np

# To get this path execute:
#    $ which ffmpeg
FFMPEG_BIN = '/home/bojan/anaconda3/envs/my-env/bin/ffmpeg'


# To find allowed formats for the specific camera:
#    $ ffmpeg -f v4l2 -list_formats all -i /dev/video3
#    ...
#    [video4linux2,v4l2 @ 0x5608ac90af40] Raw: yuyv422: YUYV 4:2:2: 640x480 1280x720 960x544 800x448 640x360 424x240 352x288 320x240 800x600 176x144 160x120 1280x800
#    ...

def run_ffmpeg(fifo_path):
    ffmpg_cmd = [
        FFMPEG_BIN,
        '-i', '/dev/video2',
        '-video_size', '640x480',
        '-pix_fmt', 'bgr24',        # opencv requires bgr24 pixel format
        '-vcodec', 'rawvideo',
        '-an','-sn',                # disable audio processing
        '-f', 'image2pipe',
        '-',                        # output to go to stdout
    ]
    return subprocess.Popen(ffmpg_cmd, stdout = subprocess.PIPE, bufsize=10**8)

def run_cv_window(process):
    while True:
        # read frame-by-frame
        raw_image = process.stdout.read(640*480*3)
        if raw_image == b'':
            raise RuntimeError("Empty pipe")
        
        # transform the bytes read into a numpy array
        frame =  np.frombuffer(raw_image, dtype='uint8')
        frame = frame.reshape((480,640,3)) # height, width, channels
        if frame is not None:
            cv2.imshow('Video', frame)
        
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
        process.stdout.flush()
    
    cv2.destroyAllWindows()
    process.terminate()
    print(process.poll())

def run():
    ffmpeg_process = run_ffmpeg()
    run_cv_window(ffmpeg_process)

run()

Et voila! I got the camera capture from Python notebook thanks to ffmpeg and OpenCV.



16 comments:

Bernard Ryan said...

Need The To Hire A Hacker❓ Then contact PYTHONAX✅

The really amazing deal about contacting PYTHONAX is that the Hack done by us can’t get traced to you, as every Hacking job we do is strongly protected by our Firewall. It’s like saying if anyone tries to trace the Hack, it will lead them to us and we block whatever actions they are doing.

We have been Invisible to Authorities for almost a decade now and if you google PYTHONAX, not really about us comes out, you can only see comments made by us or about us.

Another Amazing thing to you benefit from Hiring our Hackers is that you get a Legit and the best Hacking service, As we provide you with Professional Hackers who have their Hacking Areas of specialization.
We perform every Hack there is, using special Hacking tools we get from the dark web.

Some list of Hacking Services we provide are-:
▪️Phone Hacking & Cloning ✅
▪️Computer Hacking ✅
▪️Emails & Social Media Account Hacking✅
▪️Recovering Deleted Files✅
▪️Tracking & Finding People ✅
▪️Hunting Down Scammers✅
▪️Hack detecting ✅
▪️Stealing/Copying Files & Documents From Restricted Networks and Servers ✅
▪️Credit Score Manipulation ✅
▪️ Deleting Criminal Records✅
▪️Bitcoin Multiplication✅
▪️Binary Option Money Recovery ✅
▪️Scam Money Recovery✅ And lots more......

✳️ SPECIAL HACKING SERVICES-: we also specialize in Scam Bounty, as we chase down SCAMMERS and help individuals RECOVER Money stolen from them by this online SCAMMERS. Please be watchful about this SCAMMERS. They post ❌ENTICING TESTIMONIES and it quite Convincing.


Whatever Hacking service you require, just give us an Email to the Emails Address provided below.
pythonaxhacks@gmail.com
pythonaxservices@gmail.com

PYTHONAX.
2020 © All Right Reserved.

Tech Institute said...

Fantastic article with valuable information found very useful looking forward for next blog thank you.
Data Science Course in Bangalore

khadi Oman said...

Tried a lot of hackers but i strongly recommend thehackerspro.com where i eventually hired the dreaded darkhat hacker vladimir kolarov
although he was much more expensive than other hackers i contacted but he did a good job and their customer support on the website is friendly and very helpful.
These are ethical hackers that’ll solve all your problems like hack emails,Facebook, Twitter ,Instagram , change grades ,
erase criminal records, credit and debit top up, insurance paper,
access or recover lost files, background checks on individuals and organizations
or monitor cheating spouses’ phone or social media activities, contact at h4ckerspro@gmail.com or call/text 1 646 948 8125.
for faster and direct access just add him on telegram @h4ckerspro

doron said...

i was lost with no hope for my wife was cheating and had always got away with it because i did not know how or

always too scared to pin anything on her. with the help a friend who recommended me to who help hack her phone,

email, chat, sms and expose her for a cheater she is. I just want to say a big thank you to

SUPERIOR.HACK@GMAIL.COM . am sure someone out there is looking for how to solve his relationship problems, you can also contact him for all sorts of hacking job..he is fast and reliable. you could also text +1 213-295-1376(whatsapp) contact and thank me later

Ink Drop said...

Buy Real Driver's License Online

Email Us:- inkdrop121@gmail.com
We produce Unique Real Registered Documents with Best Quality Novelty.
All our documents are undetectable and will pass all Ultraviolet light Test
such documents includes:
-Passport
-Drivers License
-National Identification Card
-Resident Permit
-Visa/Invitation
-Diploma Certificate
-Bank Statement and many more.....

Email Us:- inkdrop121@gmail.com


What's App Number +44 7520660907

https://www.kemiio.com/shop/buyrealdriverslicenseonline

Gregor Mullins said...

Never met any hacker as discreet and fast like this Whitecollar Hackers. They are called WhiteHats and they has helped me in multiple ways first was when my ex wife cheated on me and now they are helping me paying my credit cards debts. They also help upgrading my son’s grades directly from the school portal I must say he’s the best at what he do and I recommend him to the world. Email; "Whitehatspytech@cyberservices.com , you can be sure to thank me later.

Samuel said...

My name is Samuel, I was in love with my wife and we were married for eight years with a son his is Liam,I loved my wife so much she had access to all my bank account and even my cash app which my accountant agreed to and said it was a great idea, then it took my wife and my accountant two month to get hold of all my properties,all accounts but I had a cash app which they knew nothing about,I was thrown out of my own house was sleeping in a hotel for weeks she also took possession of my son could only see him once a week then I found out she was in love with my accountant all these while so I went online and I came across a private investigator who help me get all my properties and my accounts back even my company back how he did these I don’t know but I gave all the information he asked for and followed all his instructions and now I’m happy my life’s better now.
Thanks to premiumhackservices@gmail.com
I just said I should share my own story here
Thank you

Samantha said...

I will strongly love to recommend the services of the best team of darkweb hackers. they are professional and very discreet in caryying out their jobs, they have the best customer service agents and sstisfaction at heart. If you have any services you wish to contact them for, go on alienmanhackers.xyz.
They help track and monitor your cheating partner's phone without his idea,
clear or erase criminal records as well as repair a bad credit score,
all social media hacks,funds recovery and many others.
send a mail to alienmanh4cck@protonmail.com or telegram via @alienm4nhackers.

Mills Dachin said...

CRYPTO ACCOUNT TAKEOVER (ATO) & FAKE INVESTMENT ARE ON THE RISE. attacks are on the rise and they are costing individuals, businesses, and organizations significant financial and damage that are often difficult to recover quickly. When it comes to Binary Options, there are quite people who have been taken for a ride by a Brokers and at a result of this many have lost a large amount of money to Fake Binary Option Scammers

Cybercriminals use stolen credentials such as usernames and passwords obtained by malware and social engineering to gain sensitive information, and they’re using that same data to access websites and banks/Bitcoin accounts wallet to transfer money, execute fraudulent transactions and bring people down to a Zero point financially.

D-hackers is a multinational equipped Hackers come together as a team to track down & to recover whatever that has being stolen from you from the most difficult internet SCAMMERS. NOTE!! We've received countless heartbreaking reports of notorious cyber scammers and we’ve successful recover them back.

contact us on
1⃣Binary Recovery.
2⃣Files Recovery
3⃣School Graded & Exam Questions
4⃣Password Bypass / Recovery
5⃣Malware Removal / Erase Criminal Records
6⃣Blank ATM Card
7⃣Social Media Hack
8⃣Remote Mobile Monitoring & Hacking
9⃣Clear Bad Credit
🔟Private Key Reset

Relate whatever it is to City Center Of Binary Option Service & allow us give you positive result with our hacking skills. Visit our BLOG page D-hacker.banaryoptionsport.com
Email 📩 binaryoptionservice01@gmail.com pointekhack@gmail.com cyberhackertap@gmail.com we Guarantee you up to %85
REMEMBER YOUR HAPPINESS IS OUR PRIDE

Mills Dachin said...

CRYPTO ACCOUNT TAKEOVER (ATO) & FAKE INVESTMENT ARE ON THE RISE. attacks are on the rise and they are costing individuals, businesses, and organizations significant financial and damage that are often difficult to recover quickly. When it comes to Binary Options, there are quite people who have been taken for a ride by a Brokers and at a result of this many have lost a large amount of money to Fake Binary Option Scammers

Cybercriminals use stolen credentials such as usernames and passwords obtained by malware and social engineering to gain sensitive information, and they’re using that same data to access websites and banks/Bitcoin accounts wallet to transfer money, execute fraudulent transactions and bring people down to a Zero point financially.

D-hackers is a multinational equipped Hackers come together as a team to track down & to recover whatever that has being stolen from you from the most difficult internet SCAMMERS. NOTE!! We've received countless heartbreaking reports of notorious cyber scammers and we’ve successful recover them back.

contact us on
1⃣Binary Recovery.
2⃣Files Recovery
3⃣School Graded & Exam Questions
4⃣Password Bypass / Recovery
5⃣Malware Removal / Erase Criminal Records
6⃣Blank ATM Card
7⃣Social Media Hack
8⃣Remote Mobile Monitoring & Hacking
9⃣Clear Bad Credit
🔟Private Key Reset

Relate whatever it is to City Center Of Binary Option Service & allow us give you positive result with our hacking skills. Visit our BLOG page D-hacker.banaryoptionsport.com
Email 📩 binaryoptionservice01@gmail.com pointekhack@gmail.com cyberhackertap@gmail.com we Guarantee you up to %85
REMEMBER YOUR HAPPINESS IS OUR PRIDE

Kathy Barnett Cox said...

Having a low credit score is really bad which affected me in so many ways but all thanks to you anonymousmaskhat@gmail.com for helping me in improving my credit score am really happy and also feel relived all this were done at a very affordable price incase you need similar services from anonymousmaskhat@gmail.com tell him I referred you.

micheal pan said...

BE SMART AND BECOME RICH IN LESS THAN 3DAYS....It all depends on how fast 
you can be to get the new PROGRAMMED blank ATM card that is capable of
hacking into any ATM machine,anywhere in the world. I got to know about 
this BLANK ATM CARD when I was searching for job online about a month 
ago..It has really changed my life for good and now I can say I'm rich and 
I can never be poor again. The least money I get in a day with it is about 
$50,000.(fifty thousand USD) Every now and then I keeping pumping money 
into my account. Though is illegal,there is no risk of being caught 
,because it has been programmed in such a way that it is not traceable,it 
also has a technique that makes it impossible for the CCTVs to detect 
you..For details on how to get yours today, email the hackers on : (
atmmachinehackers1@gmail.com ). Tell your 
loved once too, and start to live large. That's the simple testimony of how 
my life changed for good...Love you all ...the email address again is ;
atmmachinehackers1@gmail.com

mike said...

I just have to introduce this hacker that I have been working with him on getting my credit score been boosted across the Equifax, TransUnion and Experian report. He made a lot of good changes on my credit report by erasing all the past eviction, bad collections and DUI off my credit report history and also increased my FICO score above 876 across my three credit bureaus report. Email him here via Email him here via hackintechnology@cyberservices.com or whatsapp Number: 213 295 1376.

Grace Charlotte said...

I got a blank ATM card to withdraw $5k daily for six months. I am so happy about this because i got the cloned atm card after i was scammed twice by fake hackers. I have used the card to withdraw $420k so far. Georg Bednorz Hackers is giving out the cards to help the poor and needy though it is illegal but it is something nice. No one gets caught when using this card. you can get it from Georg Bednorz Hackers today! He is kind, lovely, humble. Also, Georg Bednorz is an expert crypto trader through his trading signals, i made $1 million bitcoins, doubling my profit in 3 weeks.
Contact Email: georgbednorzhackers@gmail.com
WhatsApp/Text: +12623558285

perez turner said...



GET RICH WITH THE USE OF BLANK ATM CARD FROM
(besthackersworld58@gmail.com)
Has anyone here heard about blank ATM card? An ATM card that allows you to withdraw cash from any Atm machine in the world. No name required, no address required and no bank account required. The Atm card is already programmed to dispense cash from any Atm machine worldwide. I heard about this Atm card online but at first i didn't pay attention to it because everything seems too good to be true, but i was convinced & shocked when my friend at my place of work got the card from guarantee Atm card vendor. We both went to the ATM machine center and confirmed it really works, without delay i gave it a go. Ever since then I’ve been withdrawing $1,500 to $5000 daily from the blank ATM card & this card has really changed my life financially. I just bought an expensive car and am planning to get a house. For those interested in making quick money should contact them on: Email address : besthackersworld58@gmail.com or WhatsApp him on +1(323)-723-2568

Anonymous said...

We are most reliable and efficient supplier/distributor of all kinds of commonly used pain killers, anxiety meds, depression meds, sleeping/sex aid pills with safe and quick delivery to any part of the world without custom issues.. We do worldwide shipping to any clear address.
Customer satisfaction is our greatest concern. Shipping is 1-3 working days depending on your country.
For more inquiries on price, purchase and shipping, get back to us at; reimannbrigit.9@gmail.com or whatsApp:+1(281) 393-7628.

Get all illicit and illegal drugs delivered to you all over the world and anywhere you want; We have the below drugs and more in stock;
Heroin, Adderall,
Cocaine,
Marijuana/Cannabis,
MDMA,
PercocetNorco 325 mg / 5 mg,
Vicodin 500 mg / 5 mg,
Viagra 100 mg,
Xanax 2 mg longbars,
Seconal (Secobarbital Sodium Capsules) 100 mg,
Nembutal (In pill, liquid and powder form),
Quaalude (lemmon 714 300mg),
Ketamine Crystal,
Dilaudid 8 mg,
Adderall 30 mg,
Oxycontin 20 mg / 80 mg,
Klonopin 0.5 mg,
Ritalin 10 mg,
Actavis: promethazine/codeine syrup,
Ambien ( Zolpidem, Stilnox) 10 mg,
Clonazepam 2 mg ( Rivotril),
Lorazepam 2.5 mg ( Ativan),
Roxicodone 15 mg, 30 mg,
Hydro 10 mg / 500 mg,
Percocet 10/325mg,
Opana 10 mg / 40 mg,
Subutex - Suboxone,
Fentanyl patches,
Diazepam Roche 10mg,
Alprazolam/Xanax 0.5mg,
Zopiclone 7.5mg,
Diazepam Shalina 10mg,
Diazepam Teva 5mg,
Zolpidem 10mg,
Tramadol 200mg,
Ketamine liquid 50mg/10ml,

Our services are as seen below;
- Stealth Packages
- Vacuum sealed thick plastic
- No prescription required
- Express prior overnight or next day delivery
- Fast and Reliable delivery #Tracking Available!
- Packing is highly discreetly sealed, wrapped with an aluminum foil, safe and secure
- No signature required upon arrival of parcel
- 100% customer care and support.
- Money back guarantee
- Offers Reshipment if package do not gets to location
- We offer door to door delivery services
* We are reliable, and we offer the best prices and services for our products.
* We offer safe and discreet overnight shipping (24/h) to clients within the
USA, and 3-4 business days shipping to clients anywhere in the world.
* We also offer the best discount for bulk purchase on any of our products.
* We offer the best of services to all our clients, and make sure they are treated like family.
* We offer a FULL REFUND on any package that doesn't make it to its destination.
Contact us:
WhatsApp: +1(281) 393-7628
Email: reimannbrigit.9@gmail.com