Porting My Python 3 Scraper Script Over to my Kali Linux VM

This is republished from my old Blogger blog but I can’t find the original Python script blog article. I’ll be back soon to post the content for the original article if I’ve kept good enough notes. Argh!

I didn’t know what else to call the title but this is what I’m doing…

I wrote a Python scraper script, see my previous blog posts on how to do this yourselves.

So I’m a little further down the road in my training and want to take that script written on my Windows PC using Python 3 and run it on a Kali Linux VM which apparently is running Python 2.7.

So as with all things tech, it won’t run without some tweaking.

This is how I got it to work.

First, there is a difference in how you execute Python scripts in Linux. With Kali Linux, you seem to have several versions installed by default. Looks like python 2.7 is the directory where most of the goodies including the BeautifulSoup package my scraper needs were so I’m using this one.

Example of how to call python2.7 then pass the script that is located in the “Desktop” folder to it.

The only code I had to update one line of  code and that was the reference to the urllib.

Why? Because the urllib and urllib2 packages have been split into urllib.request , urllib.parse and urllib.error packages in Python 3.x. The latter packages do not exist in Python 2.x

We are going backwards in version and

Old code: from urllib.request import urlopen as uReq

New code: from urllib2 import urlopen as uReq

How to Get Started Programming in Python / Python Crash Course

Crash Course in to Python Programming

I had really been wanting to find a couple of hours to do a crash course on python and finally found an opportunity to do it last night.

Having been a programmer for a while, I’m finding many of the Python videos quite boring as they are for absolute beginners which has you plodding through the basics of programming which makes learning a new programming language painfully slow.

Luckily, I had found this great Python primer video by Derek Banas on YouTube.

I followed along and completed all the code in the video and had working examples of most of the important code snippets for much of how stuff gets done in Python.  Success was not without some pain, see problems and  resolutions below.

Video: Learn Python in One Video

I will save you some pain by telling you there is a very frustrating code issue at the end of this long video.

See code fix below.

When you are in the Dog class referencing the Animal super class from which it inherits the name, weight, height and sound, properties or values you have to use the setter and getter class methods instead of the object “self” reference like we could in the Animal object.  Once you change that, the code works.

For the Dog class, the toString() method only worked for me the example below:

def toString(self):
        return "{} is {} cm tall and {} kilograms, says {} and owner is {}".format(
                                                                    self.get_name(),
                                                                    self.get_height(),
                                                                    self.get_weight(),
                                                                    self.get_sound(),
                                                                    self.__owner)
...instead of:
(this doesn't)
def toString(self):
        return "{} is {} cm tall and {} kilograms, says {} and owner is {}".format(
                                                                    self.__name,
                                                                    self.__height,
                                                                    self.__weight,
                                                                    self.__sound,
                                                                    self.__owner)

As a bonus I would add watching this video to help solidify the idea of polymorphism if you’re having difficulty getting a grasp of it.

Hope this helps someone!

Video: What is Polymorphism?