Talk About Quality

Tom Harris

Learning a New Programming Language, with Life (part 4)

leave a comment »

While preparing a course syllabus on coding in Python, for an upcoming high school class, I remembered static analysis. I haven’t decided how soon to introduce the topic, but I thought I’d better check how my own sample program fared.

No warnings or errors from Python itself. (If there had been I would have addressed them already!).

So I installed Pylint and tried its default settings. How about that. Not zero at all!

In fact, 54 coding convention messages, 12 warnings, and 1 recommendation for refactoring.

It was easy enough to clean up whitespace issues (helped to turn on “view whitespace” in my Notepad2).

And yes, many comment lines were too long. I left-justified (but indented for Python) for easy reading.

Some of the warnings are for my “TODO” comments — an extra reminder to do, or drop, next steps I’d identified earlier. Pylint message count is now down to 2 informational messages, 28 coding convention, 13 warnings, and 1 to refactor. They are valuable for my future Python learning (and teaching):

Locally disabling unused-variable (W0612) (locally-disabled)

I disabled those warnings because I know the code needs those variables. But I’ll have to explain why.

(Pylint doesn’t forget — the 2 new informational messages remind that I’ve suppressed two warnings.)

Invalid attribute name “maxRowMain” (invalid-name)

Not just that name. Most of my object names. I’ll have to find a good object-naming convention and use it.

R: 69, 4: Too many branches (13/12) (too-many-branches)

Just today I heard a Python lecturer on YouTube say, “if you’re not refactoring, you’re not learning.” Yes, that function is the longest. Not so complex, but could be simpler and easier to understand.

Attribute ‘maxRowMain’ defined outside __init__ (attribute-defined-outside-init)

A few of those also. I will have to go back and learn again about __init__: when to use it and why.

All in all, a good learning session, and direction on what I need to learn next.

Thanks to static analysis with Pylint.

Written by Tom Harris

September 2, 2015 at 10:30 pm

Learning a New Programming Language, with Life (part 3)

with one comment

I started learning Python three days ago, as described here in part 1 and part 2.

Just two hours a day, and I’m well on my way, because here below is a working implementation of my favorite, Conway’s Game of Life, in Python.

Today I refactored to put instance-specific initializations right in the functions that needed them, and promoted parsing of command line arguments to the main program. (Future: use argparse.)

I also realized that these functions are just procedures, so I stopped trying to return meaningful values. I removed the return statements. Python supports that, and returns None.

Update:

A day later, after questions from a reader led me to review my own code, I wondered why it works even without wraparound for negative indices!

The answer: Python lists handle negative indexes gracefully.

Let’s go to the code

#Life.py

#Implementation of Conway’s Game of Life
#See https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life

#Tom Harris 22-Jul-2015
#Tom Harris 23-Jul-2015 add multiple generations and display period to main program

#Import libraries needed
import sys #for command line argument support

#Define the Life class

class Life:
….#class attributes
….#None

….#class methods
….#NOTE: Currently all procedures — no return statement, implicitly returns None
….
….def readBoardFromFile(self, filename):
……..#Initialize (clear) the board of Life cells
……..self.mainBoard = []

……..#Open the file
……..initialGenerationFile = open(filename)

……..#Read from the file, line by line:
……..for line in initialGenerationFile:
…………listLine = list(line.rstrip(‘\r\n’))
…………self.mainBoard.append(listLine) #add the line, in Python list format, to the mainBoard

……..#Determine dimensions of the board after reading in
……..#WARNING: dimensions are 1 larger than largest index, since indices start at 0
……..self.maxRowMain = len(self.mainBoard)
……..self.maxColMain = len(self.mainBoard[0])

……..#Close the file
……..initialGenerationFile.close

….def displayBoard(self):
……..for rowIndex, row in enumerate(self.mainBoard):

…………for colIndex, col in enumerate(row):
…………….if self.mainBoard[rowIndex][colIndex]== “1”:
………………..print(“X”, end=”)
…………….else:
………………..print(“.”, end=”)

…………print() #print newline at end of each row

….def makeBoardCurrent(self): #TODO: Think of a better name — means to copy from the next-generation workingBoard to mainBoard
……..self.mainBoard = self.workingBoard
……..
….def calculateNextGeneration(self):
……..#Calculate next generation on workingBoard based on mainBoard

……..#Initialize work area
……..self.workingBoard = [] #A work area for creating the next Life generation

……..#Calculate next generation on workingBoard based on mainBoard
……..for mainRowIndex, mainRow in enumerate(self.mainBoard):

…………#Initialize working row before using for each row
…………workingRow = [] #for building up workingBoard row by row

…………for mainColIndex, mainElement in enumerate(mainRow):

…………….#Restart count of live cells
…………….numLiveCells = 0

…………….for i in [-1, 0, 1]:
………………..#Check row before, same row, next row
………………..rowOfCellToCheck = mainRowIndex + i

………………..#Wraparound
………………..if rowOfCellToCheck (self.maxRowMain – 1):
……………………rowOfCellToCheck = 0
……………………
………………..for j in [-1, 0, 1]:
……………………#Check column before, same column, next column
……………………colOfCellToCheck = mainColIndex + j

……………………#Wraparound
……………………if colOfCellToCheck (self.maxColMain – 1):
……………………….colOfCellToCheck = 0

……………………#If cell to check is alive, increment count of live cells
……………………#But if same row and column, ignore
……………………sameColAndRow = abs(i) + abs(j)
……………………if sameColAndRow != 0:
……………………….if self.mainBoard[rowOfCellToCheck][colOfCellToCheck] == “1”:
………………………. numLiveCells = numLiveCells + 1
…………………………………………

…………….if self.mainBoard[mainRowIndex][mainColIndex] == “1”: # The survival rules

………………..if 2 <= numLiveCells <= 3:
……………………workingRow.append(‘1’)
………………..else:
……………………workingRow.append(‘0’)

…………….else: #The reproduction rule

………………..if numLiveCells == 3:
……………………workingRow.append(‘1’)
………………..else:
……………………workingRow.append(‘0’)
……………………
…………self.workingBoard.append(workingRow)
…………………………..
#The actual main Life program
#Usage will be Life(filename, generations to calculate, every how many generations to display, display yes/no, write to files yes/no)
#First Usage Life() — done 22-Jul 10:24
#Second Usage Life(filename) — reading in file done 22-Jul-2015 10:59
#Third Usage Life(filename) — including displaying the file and dummy next generation 22-Jul-2015 18:34
#Fourth Usage Life(filename) — real next generation done 22-Jul-2015 20:17
#Fifth Usage Life(filename, generations to calculate, every how many generations to display) done 23-Jul-2015 20:44
#NOTE: Decided not to implement writing single generations to separate files. Seems unnecessary and would just fill my disk with lots of files.
#NOTE: Piping console output to text file is fine. Modern text editors can open large, multi-generation Life files and navigate quickly
#TODO: Next learning topics (backlog) would be:
#….1. Graphic display
#….2. Mouse input of boards
#….3. Change to argparse for robust command line parsing
#….4. Clean up file layout according to some accepted Python style guide
#….5. Refactor to be more functional as opposed to procedural, and reduce global variable use, if that makes code clearer

#Create a single Life instance
myLife = Life()

#Get the command line arguments
#TODO:Switch to Python argparse module instead

#From first command line argument, get pathname of initial (zero’th) generation text file
initialGenerationFilename = sys.argv[1]

#From second command line argument, get number of generations to calculate
numGenerations = int(sys.argv[2])

#From third command line argument, get period: every how many generations to display or write to file
#NOTE: currently, writing results to a file is not yet included
periodOfDisplay = int(sys.argv[3])

#Read in the initial (zero’th) generation board from text file
myLife.readBoardFromFile(initialGenerationFilename)

#Display initial (zero’th) generation
print(“Generation #”,0)
myLife.displayBoard()
print()

#Generate and display following generations
for generationIndex in range(1,numGenerations+1): #NOTE:Add 1 because range end is always one less than value
….myLife.calculateNextGeneration()
….myLife.makeBoardCurrent()
….if (generationIndex % periodOfDisplay) == 0: #NOTE: “%” is Python modulus — checking for evening divisible by period of display
……..print(“Generation #”,generationIndex)
……..myLife.displayBoard()
……..print()

And here’s some output. The 4-generation-period “glider” on a 25 x 25 grid, starts its flight from upper-left towards lower-right.
The command I ran was

Life.py Glider25by25.txt 8 4

and the output was

Generation # 0
..X......................
...X.....................
.XXX.....................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................

Generation # 4
.........................
...X.....................
....X....................
..XXX....................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................

Generation # 8
.........................
.........................
....X....................
.....X...................
...XXX...................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................
.........................

Written by Tom Harris

July 24, 2015 at 12:48 am

Learning a New Programming Language, with Life (part 2)

with one comment

I started learning Python two days ago, as described in part 1.

Yesterday I learned about Python Classes and Functions, including class and instance variables, and the special __init__ function. Today I learned enumerate, and the list function rstrip. I also learned not to forget the colon (:) in def, for, if, and else constructs!

After just a few hours today, my first project has actually started working.

For one “generation”, according to the rules of Conway’s Game of Life.

Here it is, complete with TEST comments preceding debugging code, and TODO comments for what’s next.

#Life.py

#Implementation of Conway's Game of Life
#See https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life

#Tom Harris 22-Jul-2015

#Import libraries needed
import sys #for command line argument support

#Global functions
def initialGenerationFilename():
….#Get pathname from first command line argument
….return sys.argv[1]

#Define the Life class

class Life:
….#class attributes
….#None

….#class methods

….def __init__(self):
……..#instance attributes: define and initialize (some more later after reading in file)
……..self.mainBoard = [] #The board of cells
……..self.workingBoard = [] #A work area for creating the next Life generation
….
….def readBoardFromFile(self, filename):
……..#Open the file
……..initialGenerationFile = open(filename)

……..#Read from the file, line by line:
……..for line in initialGenerationFile:
…………listLine = list(line.rstrip(‘\r\n’))
…………self.mainBoard.append(listLine) #add the line, in Python list format, to the mainBoard

……..#Determine dimensions of the board after reading in
……..#WARNING: dimensions are 1 larger than largest index, since indices start at 0
……..self.maxRowMain = len(self.mainBoard)
……..self.maxColMain = len(self.mainBoard[0])

……..#Close the file
……..initialGenerationFile.close

……..return self.mainBoard

….def displayBoard(self):
……..for rowIndex, row in enumerate(self.mainBoard):

…………for colIndex, col in enumerate(row):
…………….if self.mainBoard[rowIndex][colIndex]== “1”:
………………..print(“*”, end=”)
…………….else:
………………..print(“.”, end=”)

…………print() #print newline at end of each row

……..return 0 #TODO: replace with returning a result — what do we do with it? This is a function, not a procedure.

….def makeBoardCurrent(self): #TODO: Think of a better name — means to copy from the next-generation workingBoard to mainBoard
……..self.mainBoard = self.workingBoard
……..
……..return self.mainBoard

….def calculateNextGeneration(self):

……..#Calculate next generation on workingBoard based on mainBoard
……..workingRow = [] #for building up workingBoard row by row

……..#Calculate next generation on workingBoard based on mainBoard
……..for mainRowIndex, mainRow in enumerate(self.mainBoard):
…………for mainColIndex, mainElement in enumerate(mainRow):

…………….#Restart count of live cells
…………….numLiveCells = 0

…………….for i in [-1, 0, 1]:
………………..#Check row before, same row, next row
………………..rowOfCellToCheck = mainRowIndex + i

………………..#Wraparound
………………..if rowOfCellToCheck < 0:
……………………rowOfCellToCheck = self.maxRowMain – 1

………………..if rowOfCellToCheck > (self.maxRowMain – 1):
……………………rowOfCellToCheck = 0
……………………
………………..for j in [-1, 0, 1]:
……………………#Check column before, same column, next column
……………………colOfCellToCheck = mainColIndex + j

……………………#Wraparound
……………………if colOfCellToCheck < 0:
……………………….colOfCellToCheck = self.maxColMain – 1

……………………if colOfCellToCheck > (self.maxColMain -1):
……………………….colOfCellToCheck = 0

……………………#If cell to check is alive, increment count of live cells
……………………#But if same row and column, ignore
……………………sameColAndRow = abs(i) + abs(j)
……………………if sameColAndRow != 0:
……………………….if self.mainBoard[rowOfCellToCheck][colOfCellToCheck] == “1”:
………………………. numLiveCells = numLiveCells + 1
…………………………………………

…………….if self.mainBoard[mainRowIndex][mainColIndex] == “1”: # The survival rules

………………..if 2 <= numLiveCells <= 3:
……………………workingRow.append(‘1’)
………………..else:
……………………workingRow.append(‘0’)

…………….else: #The reproduction rule

………………..if numLiveCells == 3:
……………………workingRow.append(‘1’)
………………..else:
……………………workingRow.append(‘0’)
……………………
…………self.workingBoard.append(workingRow)
…………workingRow = []
………………..
……..return self.workingBoard

#The actual main Life program
#Usage will be Life(filename, generations to calculate, every how many generations to display, display yes/no, write to files yes/no)
#First Usage Life() — done 22-Jul 10:24
#Second Usage Life(filename) — reading in file done 22-Jul-2015 10:59
#Third Usage Life(filename) — including displaying the file and dummy next generation 22-Jul-2015 18:34
#Fourth Usage Life(filename) — real next generation done 22-Jul-2015 20:17
#TODO: Fifth Usage Life(filename, generations to calculate, every how many generations to display)

#Create a single Life instance
myLife = Life()

#Run the methods in the right order
myLife.readBoardFromFile(initialGenerationFilename())
myLife.displayBoard() #TEST: display the mainBoard before generating
print()
myLife.calculateNextGeneration()
myLife.makeBoardCurrent()
myLife.displayBoard()

And here’s the output. It’s the simplest oscillator — the “Blinker”, one generation, with wraparound:

........
*.......
*.......
*.......
........
........
........
........
........

........
........
**.....*
........
........
........
........
........
........

Written by Tom Harris

July 22, 2015 at 8:48 pm

Learning a New Programming Language, with Life

with 2 comments

For an upcoming course to help science students master the computer as a tool for their work, by learning to code, I need to learn Python. I reviewed discussions on Python 2 vs Python 3, and chose the latter.

To operate in a new language, I only need to learn and practice four things:

  1. Input
  2. Storage and Retrieval
  3. Processing
  4. Output

(Wish I could program functionally, and avoid #2. But Python isn’t designed for functional programming.)

My traditional project for learning a new language is to code Conway’s Game of Life.

After my first 2 hours in Python, with Python 3.4.3 installed, and Google for answering my questions, I’ve covered command line arguments (though still without argparse for user error handling), input from text file, output to console, line-by-line storage, and character-by-character retrieval from a Python list.

I’ve also imported one Python module (sys), and gotten used to code blocks via indentation.

Here’s my code so far. Wish me luck as I continue learning!

#ReadTextFromFileAndDisplay.py

#Import libraries needed
import sys #for command line argument support

#Get pathname of file to open from first command line argument
filenameToOpen = sys.argv[1]

#Open the file
myFile = open(filenameToOpen)
print("Pathname was: ", filenameToOpen)
print("Result code for opening the file was: \n", myFile)

#Read and display entire file
#Also store it in a list

myList = [] #declare list, initially empty

print("\nHere is the file as it's being read in: \n")
#Read from the file, line by line:
for line in myFile:
....print(line, end='') #display each line
....myList.append(line) #add the line to the list

#Close the file
myFile.close

print("\nFile should be closed now.\n")

print("Here's the contents of the list... \n")

#Display the contents of the list
for row in myList:
....print(row, end='') #entire rows
....print()
....for iCharacterIndex in range(len(row)):
........print(row[iCharacterIndex]) #character by character

p.s. I’ve put periods in to show the indentation, since it’s hard to get WordPress to preserve whitespace.

Written by Tom Harris

July 20, 2015 at 7:44 pm

If you can’t explain it simply

leave a comment »

explain_simply_prove_understanding

Written by Tom Harris

March 7, 2015 at 10:50 pm

Posted in Teaching

Security: Small is Beautiful

with 2 comments

A Story

Imagine a new procedure for your next group event. First, I tell you that you have too much stuff, so leave all your bags — backpacks, jackets, etc. — on the sidewalk outside. Then I round up some friendly volunteers off the street, and pay them minimum wage to watch your belongings. Finally, I gather some more volunteers and send one to each of your homes to browse around while you’re not there.

With Too Much Personal Data, Security Is Broken

Sounds crazy, but this is the current situation with information security. With Moore’s Law to thank, we all have much more personal data — mostly photos and videos — than we can store and protect ourselves. So we put them “In The Cloud” on some provider’s remote server. Other personal information such as financial, medical, and government records, are stored at bank, hospital, and government data centers that don’t always do the greatest job of protecting them. (Think store credit card breaches.) Meanwhile, on the most common computer we all operate — the smartphone — we run apps written by strangers that ask permission to do just about anything on our phone. And we grant that permission.

Self-Securing to the Rescue

Is there any hope? I think there is — there has to be — but it’s not in more network firewalls or PC antivirus programs. (Still, better keep those running until better alternatives are available!)

Here’s what I see as trends for a 3+1 solution:

  1. Firewalls not per network, but per application: protecting itself against virus infiltration and data theft
  2. Every task individually sandboxed, so that no task can be hijacked to do hackers’ bidding
  3. All data encrypted so even if it’s stolen or intercepted, it’s not useful or public

All these methods are available, some even with operating system support and commercial products.

The “+1”: Authentication

With every application and file locked down from the wrong people, the rightful owners still need access. So we need reliable, personal, easy-to-use authentication as well. Biometrics seems to be the way to go, but it has to be distributed back to the owners — each of us — otherwise we’re just creating another central database waiting to be stolen.

Challenges

While these technologies are on the way, there are still challenges from competing interests and behaviors.

One is governments — even the ones trying to protect us. The first encrypted e-mail service shut down rather than submit to government surveillance, and a second one followed soon after, even without any immediate legal threats.

The other is us — businesses and clients not ready to do our part for personal information security. I could mention easy-to-guess passwords, but here’s a better example. The other day I received a personal file by e-mail from an insurance company. The file was attached, encrypted. Sounds fine, right? Except the insurance company’s software generates the e-mails with (a) the encrypted file attached; (b) the text of the unencrypted e-mail saying that the password is my id number and (c) my id number in the subject of the e-mail!

My Prediction

It’s going to take a while, and more denial of service events and data breaches, but the current situation is untenable. Governments and the information security industry, including big players and startups, will work together to develop and deploy standards, and easy-to-use systems, that delegate security to each application and file, and authentication to each legitimate user.

Until then, go outside to the sidewalk and see if your stuff is still there where you left it.


I presented these thoughts at the recent CyberJLM #3 in a 5-minute, no-slides, lightning talk format.

Written by Tom Harris

December 28, 2014 at 11:37 pm

Posted in Cybersecurity

Agile Revolution Breaks Into Business and Life

with one comment

Agile has been transforming software organizations one by one since the turn of the millenium.

Quadrupling productivity while making people and teams happier.

Modern work and life is knowledge-oriented, fast, and ever-changing. That’s what Agile addresses.

Why doesn’t anyone outside of software development adopt it?

Steve Denning, in a 2012 Forbes Magazine article, called it The Best-Kept Management Secret On The Planet.

It’s still pretty secret, but it shouldn’t be. The word “software” appears only once in the four statements of the Agile Manifesto. Three times in the Principles of Agile. Just replace “software” with “anything”.

Need confirmation? Try Ana Willem’s Should Non-technical Organizations model Agile principles? (no brainer).

So who’s doing it? Who is being Agile in their business or their life?

Here are the very few examples I’ve found so far:

Can you share more?

Written by Tom Harris

December 18, 2014 at 9:34 pm

Posted in Agile

Follow

Get every new post delivered to your Inbox.