Get keywords from YouTube video meta tag.


• March 2, 2019 • Comments Off on Get keywords from YouTube video meta tag.

Install Python 3

Install Beautiful Soup
Ubuntu:

sudo apt-get install python3-bs4
sudo apt-get install python-lxml

PIP:

python3 -m pip install BeautifulSoup4
python3 -m pip install lxml
import urllib.request
from bs4 import BeautifulSoup  

url = 'https://www.youtube.com/watch?v=BWYiMcH0cRc'

print('URL: %s' % url)
print('*' * 60)

request = urllib.request.Request(url)
response = urllib.request.urlopen(request)
soup = BeautifulSoup(response,"lxml")

keywords = soup.find("meta", {"name":"keywords"})['content']
for keyword in keywords.split(','): 
    print(keyword.strip())

Result:
URL: https://www.youtube.com/watch?v=BWYiMcH0cRc
****************************************
indoor plants
indoor plant guide
house plants

Make it a simple Tk application:

import urllib.request
from bs4 import BeautifulSoup  
from tkinter import *

def get_keywords():
    url = entry.get()
    if not url: return

    listbox.delete(0, END)
    request = urllib.request.Request(url)
    response = urllib.request.urlopen(request)
    soup = BeautifulSoup(response,"lxml")

    keywords = soup.find("meta", {"name":"keywords"})['content']
    for id,keyword in enumerate(keywords.split(',')): 
        #print(keyword.strip())
        listbox.insert(id, keyword.strip())
    entry.delete(0, END)
    entry.focus()

def close_window (): 
    root.destroy()

root = Tk()

root.title('Keywords')
root.geometry("640x480") 
root.resizable(0, 0)

topframe = Frame(root)
topframe.pack(side = TOP, fill = X)

label = Label(topframe, text = "URL")
label.pack(side = LEFT)

entry = Entry(topframe, bd = 1)
entry.pack(fill = X)
entry.focus()

buttonframe = Frame(root)
buttonframe.pack(side = BOTTOM, fill = X)

button_run = Button(buttonframe, text="Run", command=get_keywords)
button_run.pack(side = RIGHT)
button_close = Button(buttonframe, text="Close", command=close_window)
button_close.pack(side = LEFT)

bottomframe = Frame(root)
bottomframe.pack(side = BOTTOM, fill = BOTH, expand = 1)

listbox = Listbox(bottomframe)
listbox.pack(side = LEFT, fill = BOTH, expand = 1)

root.mainloop()

How to fix a ASUS ROG laptop installing Pop OS an adding HP scanning ability.


• February 1, 2019 • Comments Off on How to fix a ASUS ROG laptop installing Pop OS an adding HP scanning ability.

Installing Pop! OS 18.10 on a 2017 Asus ROG fixes every problem Windows created. 😂 Over heating? Fixed. USB connection issues? Fixed. Driver issues? None. Touch Pad sucks? Nope. Of course you might not be able to play all your favorite Windows games, but I get most to work if I put my mind to it.

Create a bootable Pop USB from the latest ISO.

Plug in the USB. Reboot into BIOs, F2 while starting.

Turn off Secure Boot and set the USB as the first boot disk option in the BIOs. Save and reboot.

Install Pop 18.10

Go into drivers and select Nvidia Proprietary. Reboot. I reverted back to Noveau since we are not gaming. Driver is solid. Ubuntu and Ubuntu Budgie causes issues with Nvidia drivers, FYI.

Check for updates in the Pop Store – automatic. Install all. This will update Nvidia settings app also.

Install Budgie, or whatever DE you want, via apt. Do not add any PPAs for Budgie. Apt will give you the latest. I intall Budgie because I like the way OS X flows and it’s very similar.

sudo apt install ubuntu-budgie-desktop

** If you need to setup the HP scanner drivers:

Install Synaptic package manager. Do a complete removal of all HP/HPLip packages OR do apt purge.

Download the driver from HP.
Search web for HP Plugins. You can try automatic install during driver install but it would not work for me. The plugin should match the driver version, currently 3.18.12.

Right click each file, select permission tab, click executable.

Run the hplip from the terminal. ./hplip.version.run

You can use defaults. When it asks for plugin point to the plugin you downloaded. Scanner should now work.
**

If you game install Steam from the Pop Store and Lutris from the Lutris site. Should be ok with the proprietary driver. If I set it up I’ll let you know.

Done. Setup the rest as you like.

See my software list for some great applications to get you on your way.

Installing Cinelerra GG on Pop or Ubuntu 18.10.


• January 20, 2019 • Leave a Comment

If you want the latest Cinelerra GG on 18.10 you’ll need to trick the system. If you’re not comfortable you can always wait.

First thing is first, add the PPA. This will cause an error, but we will fix that so ignore it. Type each command in the terminal:

sudo apt-add-repository \
https://cinelerra-gg.org/download/pkgs/ub18

Open the sources file:

sudo /etc/apt/sources.list

Find the line that references cinelerra and change it to, all on a single line:
You can just add and replace the highlighted items.

deb [trusted=yes] 
 https://cinelerra-gg.org/download/pkgs/ub18 
 bionic main

Save: Cntl+o then enter.

then:
Be aware it will take a minute to update. Let it do its thing.

sudo apt update 
sudo apt install cin

If you run into issues open the sources list again and remove the two Cinelerra lines and apt update again.

For complete removal:

sudo apt purge cin

Then remove the PPA from sources file. Delete anything related in home, like .cin or .cinelerra and the project folder if they exist.

To remove keeping cofigs:

audo apt remove cin

Linux Terminal “NCurses” Application list.


• January 18, 2019 • Comments Off on Linux Terminal “NCurses” Application list.

If you really love the terminal and can’t get enough here is a list of programs that should make you smile. Linux comes packed with so many command line tools already; you can do just about anything without the GUI, but what if you want more? Maybe some of these programs are for you.

Many programs in this list can be installed with your package manager, like apt or yum. Always worth a shot attempting to install from the package manager first.

First and foremost, get a good terminal emulator. I use Tilix as it’s solid and you can have multiple panes. I like it. Should be available already but if not: https://gnunn1.github.io/tilix-web/

On to the programs!

General

WeGo – Weather Forecasts: https://github.com/schachmat/wego

ncmpcpp – Music Manager/Player: https://rybczak.net/ncmpcpp/

cmus – Music Player: https://cmus.github.io/

gcalcli – Google Calender: https://github.com/insanum/gcalcli

Calcurse – Calender, Scheduler: https://www.calcurse.org/

Task Warrier – Task Manager: https://taskwarrior.org/

Communicate

Alpine – Mail Client: https://www.washington.edu/alpine/

Finch – Chat Client: http://developer.pidgin.im/

WeeChat – IRC: https://weechat.org/

Charm – Blog Client: http://ljcharm.sourceforge.net/

Writing

Nano – Text Editor: https://www.nano-editor.org/ (Usually installed)

Vim – Programmers Text Editor: https://www.vim.org/ (Usually installed)

Jed – Programmers Text Editor: http://www.jedsoft.org/jed/index.html

System Tools

Midnight Commander – File Manager: https://midnight-commander.org/

iftop – Network bandwith Monitor: http://www.ex-parrot.com/pdw/iftop/

htop – Process Viewer: http://hisham.hm/htop/

ncdu – Disk Usage Viewer: https://dev.yorhel.nl/ncdu

zsh – Terminal Clock: good example here

Web junk

ffsend – File Sharing: https://github.com/timvisee/ffsend

rTorrent – Torrent: https://rakshasa.github.io/rtorrent/

Canto – News Reader: https://github.com/themoken/canto-curses

w3m – Web Browser: http://w3m.sourceforge.net/

elinks – Web Browser: http://elinks.or.cz/

Games

Nethack – Dungeon Crawler: http://www.nethack.org/

Tetris: http://fph.altervista.org/prog/bastet.html

I was mostly focused on things I know or use. I don’t game in the terminal. If you do https://itsfoss.com/best-command-line-games-linux/

Execute PowerShell from Python


• January 11, 2019 • Comments Off on Execute PowerShell from Python

I originally wrote this to execute PowerShell scripts from a Flask web interface. You can now install Powershell on Linux. Crazy, ain’t it?

import subprocess
import datetime
import time
import os
import ast 
import json
# =======================================================
# Original name: lib_powershell.py
# =======================================================

class PowerShellProc():

    def __init__(self):
        self.pwrshell_path = r'C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe'
        self.station = r'\\repo\ps' # where the PS scripts live.

    def exec_pwrshell(self, pwr_script=None, script_path=None, args=[], ps_tuple_to_py=True, gm_flag=False, date2str=False):
        '''Out.exec_pwrshell('psscript.ps1' ,script_path=None, ps_tuple_to_py=True)
        script_path: overrides envrionment path.exists
        ps_tuple_to_py: Powershell tuples convert to dictionaries, set True to return a Python tuple.
        '''
        data = None
        exec_path = self.pwrshell_path
        
        if not os.path.exists(exec_path):
            raise IOError('Powershell not found at %s.' % exec_path)

        if not script_path:
            script_path = self.station
        
        if not os.path.exists(script_path):
            raise IOError('Script path not found at %s.' % script_path)

        if pwr_script:
            script_path = os.path.join(script_path, pwr_script)

        cmd = [exec_path, script_path]
        
        if args:
            if not isinstance(args, list):
                raise TypeError("Arguments must be passed as a list not %s." % str(type(args)))
            cmd.extend(args)

        proc = subprocess.Popen(cmd, 
                    shell=True,
                    stdin=subprocess.PIPE,
                    stdout=subprocess.PIPE,
                    )
        
        rtrn,perr = proc.communicate()
        rcode = proc.returncode

        #if rtrn:
            # Line directly below converts PowerShell Json to Python data structs.
            #data = self.ps_output_converstion(rtrn, tup=ps_tuple_to_py, gm=gm_flag, date2str=date2str)
        if perr:
            # Error string...
            data = json.dumps(perr.decode('utf-8','ignore'))
        else:
            #data = rtrn.decode('utf-8','ignore')
            data = self.ps_output_converstion(rtrn, tup=ps_tuple_to_py, gm=gm_flag, date2str=date2str)
        return rcode,data

    def ps_output_converstion(self, res, tup=True, gm=False, date2str=True):
        '''Converts Powershell string output to Python type'''
        
        res = res.decode('utf-8','ignore')
        
        if '\n' in res:
            res = res.replace('\n','')
        if '\r' in res:
            res = res.split('\r')[0]

        res = ast.literal_eval(res)
        
        # Data conversion if necessary.
        if type(res) == str:
            if 'date' in res.lower() or 'time' in res.lower():
                epoch_elem = elem[elem.find("(")+1:elem.rfind(")")]
                dt = self.ps_epoch_to_datetime(float(elem), gm=gm, date2str=date2str)
                res = dt
        if type(res) == list:
            for index, elem in enumerate(res):
                if 'date' in str(elem).lower() or 'date' in str(elem).lower():
                    epoch_elem = elem[elem.find("(")+1:elem.rfind(")")]
                    dt = self.ps_epoch_to_datetime(float(elem), gm=gm, date2str=date2str)
                    res.remove(elem)
                    res.insert(index,dt)
        if type(res) == dict:
            for field, value in res.items():
                if 'date' in str(value).lower():
                    epoch_elem = value[value.find("(")+1:value.rfind(")")]
                    dt = self.ps_epoch_to_datetime(float(epoch_elem), gm=gm, date2str=date2str)
                    res[field] = dt
            if tup:
                if 'Item1' in res:
                    l = [v for v in res.values()]
                    res = tuple(l)
        return res

    def ps_epoch_to_datetime(self, epoch_time, gm=False, date2str=False):
        '''Converts Powershell Date to Python datetime.'''
        if gm:
            struct = time.gmtime(epoch_time/1000.)
        else:
            struct = time.localtime(epoch_time/1000.)

        dt = datetime.datetime.fromtimestamp(time.mktime(struct))
        if date2str: dt = dt.ctime()
        return dt

    def py2json(self, data):
        return json.dumps(data) # This will convert datetime to string.

    def json2py(self, data):
        return json.loads(data)

'''
Examples of various data structures returned by a Powershell script.

>>> import lib_powershell
>>> ps = lib_powershell.PowerShellProc()
>>> ps.exec_pwrshell(pwr_script='test.ps1',script_path=r'E:\pyprod\prod_pwrshell',ps_tuple_to_py=False) # Tuple to dict.
(0, {'Item3': 1, 'Length': 3, 'Item1': 'Theresa Wilson', 'Item2': datetime.datetime(2018, 1, 1, 6, 0)})

>>> ps.exec_pwrshell(pwr_script='test.ps1',script_path=r'E:\pyprod\prod_pwrshell')
(0, (1, 3, 'Theresa Wilson', datetime.datetime(2018, 1, 1, 6, 0)))

>>> ps.exec_pwrshell(pwr_script='test.ps1',script_path=r'E:\pyprod\prod_pwrshell', gm_flag=True) # Mixed Tuple
(0, (1, 3, 'Theresa Wilson', datetime.datetime(2018, 1, 1, 11, 0)))

>>> ps.exec_pwrshell(pwr_script='test.ps1',script_path=r'E:\pyprod\prod_pwrshell')
(0, {'Oregon': 'Salem', 'Washington': 'Olympia', 'California': 'Sacramento'})

>>> ps.exec_pwrshell(pwr_script='test.ps1',script_path=r'E:\pyprod\prod_pwrshell') # Mixed array/arraylist
(0, ['a', 'b', 'c', 'd', 'e', 1, 2, 3])

>>> ps.exec_pwrshell(pwr_script='test.ps1',script_path=r'E:\pyprod\prod_pwrshell') # Int
(0, 1)

>>> ps.exec_pwrshell(pwr_script='test.ps1',script_path=r'E:\pyprod\prod_pwrshell') # String
(0, 'Hello')

# NOTE: Tuple returned is (process_return_code, powershell_return_data)
'''

List or search all installed apps on Ubuntu 18.04 with Python.


• January 10, 2019 • Comments Off on List or search all installed apps on Ubuntu 18.04 with Python.

I was bored… Of course this can be done easily in the terminal, but what’s the fun in that? This should work on any flavor of Debian/Ubuntu.

import os
import subprocess as sp
'''
Pipe to file in terminal:

$ flatpak list --app --show-details > ~/Documents/installed-flatpack.log
$ dpkg --get-selections > ~/Documents/installed-software.log

Not using Debian? Here's some others, including BSD Unix.
$ rpm -qa
$ yum list installed
$ pacman -Qi
$ pkg_version | less

Why Python when you have pipes, greps, and such?
Why not?
'''

def get_flatpak():
    cmd=['/usr/bin/flatpak','list','--app','--show-details']
    process=sp.Popen(cmd, stdout=sp.PIPE, stderr=sp.PIPE)
    stdout, stderr = process.communicate()
    return stdout.decode().replace('\t',' ').split('\n')

def get_apt():
    cmd=['/usr/bin/dpkg','--get-selections']
    process=sp.Popen(cmd, stdout=sp.PIPE, stderr=sp.PIPE)
    stdout, stderr = process.communicate()
    return stdout.decode().replace('\t',' ').split('\n')
    
if __name__=="__main__":
    
    names_only = []
    flatpak = True
    apt = True
    search_string='gimp'
    
    if flatpak:
        pkgs = get_flatpak()
        for pkg in pkgs:
            if ' ' in pkg:
                names_only.append(pkg.split()[0])
                
    if apt:
        pkgs = get_apt()
        for pkg in pkgs:
            if ' ' in pkg:
                names_only.append(pkg.split()[0])
        
    for app in names_only:
        if search_string:
            if search_string not in app.lower():
                continue
        print(app)

Create Desktop App from WebApp.


• January 10, 2019 • Comments Off on Create Desktop App from WebApp.

Here’s a nifty tool for Windows, OS X, and Linux to convert web apps, like the online Amazon Kindle Reader, to a desktop app.

https://www.npmjs.com/package/nativefier

Running Steam Windows Games on Linux


• December 30, 2018 • Comments Off on Running Steam Windows Games on Linux

I’m sure these will be hit or miss, but with a little work some Steam Windows games might run. If you want a unified gaming center on Linux with Steam for Windows support check out Lutris. Might be easier than pure Wine or Play On Linux. We will see.

Screenshot

and…

Watch Dogs 2 is running without issue on Linux!

Screenshot

Now getting the Nvidia drivers working at full steam is another matter. I am using 415 version with a GTX 1060. These can be installed from the Ubuntu PPA:

ppa:graphics-drivers/ppa

Here’s a nice write up on linuxconfig.org

Using Linux with iPhone the easy way.


• December 27, 2018 • Comments Off on Using Linux with iPhone the easy way.

Step one: Photos and video.

Most distros will mount the DCIM so you can access your photos, but I want to control them from the phone and select from there what I want to keep. So I setup a sync folder on my largest drive and setup Shotwell to watch its photos folder on the same drive. I then setup MEGA Sync, which is like Dropbox but you get 50gb free, to save to the sync folder. I wrote a small Python program to move files from the sync folder to the photos folder and set it as a task to run every 10 minutes. I turned off the camera sync in the MEGA iPhone app, otherwise it will move every photo you take. When I want photos in Shotwell I simply move the photo(s) to the MEGA app on the phone and let the cron task and Shotwell manage the files as they show up. It works great.

The code is simple, here’s an example:

#!/usr/bin/env python
import os
import shutil
 
def move_files(src_root, dest_root):

    dest_photo = os.path.join(dest_root,'Photo')
    dest_music = os.path.join(dest_root,'Music')
    dest_other = os.path.join(dest_root,'Documents')

    for root, dirs, files in os.walk(src_root):
        for _file in files:

            file_path = os.path.join(src_root, _file)

            if os.path.isfile(file_path):

                ext = os.path.splitext(_file.lower())[-1]

                if ext in ['.mov','.jpg','.png','.mp4']:
                    dest = os.path.join(dest_photo,_file)
                    shutil.copyfile(file_path,dest)
                    os.remove(file_path)
                elif ext in ['.mp3','.ogg','.flac']:
                    dest = os.path.join(dest_music,_file)
                    shutil.copyfile(file_path,dest)
                    os.remove(file_path)
                else:
                    dest = os.path.join(dest_other,_file)
                    shutil.copyfile(file_path,dest)
                    os.remove(file_path)
                    
if __name__ == "__main__":
    
    base_src = '/mnt/storage/MEGASync/'
    base_dest = '/mnt/storage'
    
    move_files(base_src, base_dest)

You could modify this script to isolate and move any file type to various folders, like if a Python file gets dropped into the Mega folder you could have the script move it to Code, or move text files to Writing. This Python code should also work on OS X and Windows. Make sure you change the paths to yours.

Add a directory named outgoing to the Megasync directory for anthing you want to pick up on the phone so it doesnt get moved. Anything there will be left alone since we are not traversing all sub directories.

Step two: Music.

Since you cannot manipulate the contents of the iTunes library on the phone I dumped Apple’s music player. I hated the ads anyway, plus I do not stream. I could install iTunes in Wine but I like Rhythmbox. I installed Flacbox on the iPhone and setup a Samba share on my music folder on my laptop. I logged into the share in Flacbox where I manage the music and playlists on my phone. I don’t mind managing phone and desktop playlists separately. Eitherway, my whole library on my machine is on the iPhone at my deposal without storing it there, unless I want to. I also researved my ip on my router so I never need to remount the share unless I change things.

Step three: App data

Install ifuse if it’s not installed.

sudo apt install ifuse

Create a folder in mnt named iPhone.

You will need the app id. Here is an example of an app named Number9 by a company named Magic Hat and how to mount its sandboxed folder:

sudo ifuse /mnt/iPhone –appid com.magichat.Number9

If successful you can browse the app’s files. This is what I am working on now. It’s a pain as I do not want to jailbreak my device and have to guess or connent to iTunes to get the ID. If there is a way on the phone I coundn’t find it.

If you need help with Samba I’m certain there are resources on the internet.

Other options?

Photos: Digikam will connect to the phone but it rarely worked for me. Alternatively Rapid Photo Downloader worked better but still hit or miss. It will also allow duplicates as will most photo managers.

Music: In Flacbox you can fire up its internal server and move music through a web browser. A little more work but it works. There are other players that will allow music imports, I think Vox will, but they are greedy.

Password Manager: If you keep Passwords in a manager you can use Keepass, but Enpass is the best. I have used it for years on Mac and now they have a Linux version that syncs with the iPhone version.

Note:

If you’re not keen to crontab in the terminal this tool makes life easy, and it works on Ubuntu 18.04. Gnome Scheduler

Compiled PDF List of the best Linux Software of 2018.


• December 23, 2018 • Comments Off on Compiled PDF List of the best Linux Software of 2018.

At least in my opinion. 🙂 Yes, it’s a PDF – I’m too lazy to sit here and format HTML.

Download as PDF

Check the hash before opening. SHA1: 65ba078feae3226c523541a26e97c3ee73c08544

Anything you think should be here or if you need help finding something, leave a comment.