README.md

@@ -14,20 +14,30 @@ Table of Contents
 
 ## SYNOPSIS
 
-The **dirnotes** family of apps allows you to add a descriptive comment to a file. The descriptions are stored in two places:
+The **dirnotes** family of apps allows you to add a descriptive comment to a file. 
+The descriptions are stored in two places:
 
  * in the _xattr_ properties of the file
  * in a _database_ located in the user's home directory
 
 [The [MacOS](#macos) stores its comments in a similar way.]
 
-The <code>**dirnotes**</code> app is a GUI app, using the Qt5 framework. At startup, it displays the contents of the current directory, and the comments associated with any of the files or directories. 
-Simple mouse clicks allow you to tunnel down into directories, or rise up the file system. 
-You can create/edit comments and choose whether the _xattr_ or _database_ version of the comments take priority, 
+The <code>**dirnotes**</code> app is a GUI app, using the Qt5 framework. 
+At startup, it displays the contents of the current directory, and the 
+comments associated with any of the files or directories. 
+Simple mouse clicks allow you to add or edit comments, tunnel down 
+into directories, or rise up the file system. 
+You can copy or move files (_with_ comments), and 
+choose whether the _xattr_ or _database_ version of the comments 
+have display priority. 
 
-The <code>**dirnotes-tui**</code> is a very similar app, but uses the _curses_ framework to display its activity in a terminal window. This can be handy if you have to work across a network, or if terminal apps are you preference.
+The <code>**dirnotes-tui**</code> is a very similar app, but uses the 
+_curses_ framework to display its activity in a terminal window. 
+This can be handy if you have to work across a network, 
+or if terminal apps are your preference.
 
-The <code>**dirnotes-cli**</code> is a command line tool, which may be handy for scripting.
+The <code>**dirnotes-cli**</code> is a command line tool, 
+which may be handy for scripting. This all can also do maintenance on the database.
 
 ## USAGE
 
@@ -40,38 +50,47 @@ _s_ for sort, _M_ to change between xattr/database priority.
 
 The **<code>dirnotes-cli</code>** has options for _-l_ list and _-c_ create a comment. 
 
-All three apps in the **dirnotes** family have the ability to copy files from the current directory. 
+All three apps in the **dirnotes** family have the ability to 
+copy/move files from the current directory, keeping the comments intact. 
+All three apps have the **-h** option which shows command line usage. 
  
 ## INSTALLATION
 
 Each of the 3 apps in the family is self contained. 
 The <code>**dirnotes**</code> app requires _Python3_ and the _Qt5_ framework. 
-The <code>**dirnotes-tui**</code> and <code>**dirnotes-cli**</code> apps simply require _Python3_.
+The <code>**dirnotes-tui**</code> and <code>**dirnotes-cli**</code> apps 
+simply require _Python3_.
+
+Simply copy the file into your path, to ~/.local/bin for example. 
+For a better GUI experience, copy 
+<code>dirnotes.desktop</code> to <code>~/.local/share/applications</code> and 
+<code>dirnotes.xpm</code> to <code>~/.local/share/icons/</code> 
 
 ### CONFIG FILE
 
-By default, the file **~/.dirnotes.conf** will be used to load the user's config. 
+By default, the file **~/.config/dirnotes/dirnotes.conf** will be used to 
+load the user's config. 
 This is a JSON file, with three attributes that are important:
 
-> * xattr_tag (default: <code>usr.xdg.comment</code>)
-> * database (default: <code>~/.dirnotes.db</code>, sensible alt: <code>/var/lib/dirnotes.db</code>) 
-> * start_mode (_xattr_ or _db_ priority)
+> * xattr_tag (default: <code>usrr.xdg.comment</code>)
+> * database (default: <code>~/.local/share/dirnotes/dirnotes.db</code>, sensible alt: <code>/var/lib/dirnotes.db</code>) 
+> * start_mode (_xattr_ or _db_ display priority)
 
 The _config_file_ should be auto-generated the first time one of the **dirnotes** apps is run.
-[_not fully implemented_]
 
 ## LIMITATIONS 
 
-
 The file comments are located in two locations: a database, and in the 
 xattr properties of the file. Each of these storage locations has its 
 own benefits and limitations. These can be summed up: **_xattr_** comments
 follow the iNode, **_database_** comments follow the file name.
 
+
 ### xattr
 
 Comments stored in the xattr properties can be copied/moved with the file, if you
-use the correct options: <code>**cp -p**</code>. The <code>**mv**</code> utility 
+use the correct options: <code>**cp&nbsp;-p&nbsp;_src&nbsp;dest_**</code>. 
+The <code>**mv**</code> utility 
 automatically preserves _xattr_. Other programs can also be coerced into 
 perserving _xattr_ properties:
 
@@ -81,8 +100,11 @@ perserving _xattr_ properties:
 
 Not all file systems support xattr properties (vfat/exfat does not).
 
+_xattr_ comments may only be applied to files for which the user has _write_ permission.
+
 The current implementation of <code>**sshfs**</code> and <code>**scp**</code> 
-do not support the copy of _xattr_ properties.
+do not support copying of _xattr_ properties. **Dropbox** type mounts are 
+unlikely to support _xattr_ comments.
 If you want to copy files to a remote machine and include the _xattr_ comments, use <code>**rsync**</code> with the _-X_ option. Or <code>**tar**</code>.
 
 Some editing apps (like _vim_) will create a new file when saving the data, which orphans the _xattr_ comments. For these apps, use the _database_ system.  
@@ -91,23 +113,30 @@ Removable disk devices (usb sticks) which are formatted with a Linux-based files
 
 ## database
 
-Comments stored in the database work for all filesystem types (including vfat/exfat/sshfs)
+Comments stored in the database work for all filesystem types 
+(including vfat/exfat/sshfs)
  
-Comments are personalized to the _current user_.
-Another user on the same system will not see these comments.
+The _database_ comments that are stored in 
+<code>~/.local/share/dirnotes/dirnotes.db</code> are inherently associated 
+with a single user. If the _database_ is located in 
+<code>/var/lib/dirnotes.db</code>, it can be shared by all the users in the system.
 
 Files are indexed by their complete path name. Removable filesystems should be
-mounted in a consistent way, so that the complete path name is reproducable.
+mounted in a consistent way, so that the complete path name is reproducable. 
+Symlinks are _not_
+dereferenced, so they may have comments bound to them.
 
 Comments stored in the database do not travel with the files when
 they are moved or copied, unless using the **dirnotes** family of tools. 
 
-The _database_ comments that are stored in <code>~/.dirnotes.db</code> are inherently associated with a single user. If the _database_ is located in <code>/var/lib/dirnotes.db</code>, it is shared by all the users in the system. The comment with the 'most recent timestamp' wins.
+_Database_ comments may be applied to any visible file, _even if they are readonly_. 
+For exmple, comments may be attached to the files in <code>/usr/bin/\*</code> even though they are probably owned by _root_.
 
 ## PROGRAMMER NOTES
 
 Instead of an API, here is how you can get directly at the underlying comment data. 
-If you intend to use the **dirnotes** apps, try to keep the two versions of the comments in sync.
+If you intend to use the **dirnotes** apps, 
+try to keep the two versions of the comments in sync.
 
   * xattr
 
@@ -122,11 +151,14 @@ to display the comments/author/date on a file. For example:
         user.xdg.comment.author: patb
         user.xdg.comment.date: 2022-09-29 08:07:42
 
-The other options on the **xattr** command line tool allow you to write (*xattr -w*) or delete (*xattr -d*) the comments.
+The other options on the **xattr** command line tool allow you to 
+write (*xattr -w*) or delete (*xattr -d*) the comments.
 
   * database
 
-The comments are stored in an Sqlite3 database, usually located at "~/.dirnotes.db". The database itself is contained within that file, and its schema is this:
+The comments are stored in an Sqlite3 database, usually 
+located at "~/.local/share/dirnotes/dirnotes.db". 
+The database itself is contained within that file, and its schema is this:
 
 ~~~~
     CREATE TABLE dirnotes (name TEXT, date DATETIME, size INTEGER, comment TEXT, comment_date DATETIME, author TEXT)
@@ -143,9 +175,12 @@ The comments are stored in an Sqlite3 database, usually located at "~/.dirnotes.
 |author     |   the system name of the user who created the comment  |  patb | 
 
 
-The _date_ and _size_ fields reflect the file's modification date and size at the time of the last edit of the file comment, which is stored in _comment_date_.
+The _date_ and _size_ fields reflect the file's modification date and size 
+at the time of the last edit of the file comment, which is stored in _comment_date_.
 
-As comments are editted or appended, new records are added to the database. Older records are are not purged. This gives you a history of the comments, but it means that fetching the most recent comment involves something like
+As comments are editted or appended, new records are added to the database. 
+Older records are are not purged. This gives you a history of the comments, 
+but it means that fetching the most recent comment involves something like
 
 ~~~~
   SELECT * FROM dirnotes WHERE name=? ORDER BY comment_date DESC
@@ -157,10 +192,24 @@ The database is created the first time one of the **dirnotes** apps is run.
 
   * misc
 
-The <code>**dirnotes**</code> gui app has a desktop icon built into the code. There is not need for an external .icon file.
+The <code>**dirnotes**</code> gui app has a desktop icon built into the code. 
+There is not need for an external .icon file, but there is an .xpm file included
+in the project, which can be copied to ~/.local/share/icons/
+
+### comment date & author
+
+The <code>copy()/move()</code> methods that are built into the **dirnotes** library
+will ask the operating system to copy/move the file _with_ xattr intact. 
+The entry in the database is created _at the time of invocation_. 
+Therefore, the xattrs will reflect the original author+date on the comments, 
+whereas the database version is updated on each copy/move; 
+the dirnotes-comments details will therefor diverge over time.
 
 There was _no_ consideration given for language translation. Email [me](mail:patb@pbeirne.com) if you want this, or can help.
 
+All these apps only accomadate a single line comment. An embedded newline will 
+cause unpredictable behaviour. 
+
 ### MacOS {#macos}
 
 The **MacOS** inherently supports file comments. The Finder app manages most of the user activity. It handles file comments in a similar manner to **Dirnotes**. Comments are stored in two places:
@@ -192,4 +241,18 @@ the three apps. And there _may_ be some inconsistency.
   The _qt-gui_ app is working pretty well.  
 
 
+QUESTIONS:
+
+There are several open-ended questions that need to be answered. 
+Does anyone have an opinion?
+
+1. How important is multi-line comments?
+
+2. Is it ok to put the config file and database file buried in ~/.config and ~/.local? 
+
+These directories exist on computers with a gui/windowing system installed, but don't neccessarily exist on headless servers. Perhaps the default locations should be in the user directory? (~/.dirnotes.conf and ~/.dirnotes.db)
+
+3. Who needs translations?
+
+4. Does anybody have a better edit-window for CURSES?
 

dirnotes

@@ -1,10 +1,8 @@
 #!/usr/bin/python3
-# TODO: get rid of sqlite cursors; execute on connection
-# TODO: add index to table creation
-# TODO: why are there TWO sqlite.connect()??
-#   try auto-commit (isolation_level = "IMMEDIATE")
-#   try dict-parameters in sql statements
-# TODO: pick up comment for cwd and display at the top somewhere, or maybe status line
+
+# TODO: XDG compatibility: config in ~/.config/dirnotes/dirnotes.conf and data in ~/.local/share/dirnotes/dirnotes.db
+# TODO: clearing a comment out to '' doesn't seem to work on both sides
+
 """ a simple gui or command line app
 to view and create/edit file comments
 
@@ -23,7 +21,7 @@ nav tools are enabled, so you can double-click to go into a dir
 
 """
 
-VERSION = "0.5"
+VERSION = "0.8"
 
 helpMsg = f"""<table width=100%><tr><td><h1>Dirnotes</h1><td>
 <td align=right>Version: {VERSION}</td></tr></table>
@@ -40,8 +38,8 @@ field of the file system.
 <p> Double click on directory names to navigate the file system. Hover over
 fields for more information.
 
-<h3>xattr extended attributes</h3>
-The xattr comment suffers from a few problems:
+<h4>xattr extended attributes</h4>
+The <i>xattr</i> comment suffers from a few problems:
 <ul>
   <li>is not implemented on FAT/VFAT/EXFAT file systems (some USB sticks)
   <li>xattrs are not (by default) copied with the file when it's duplicated 
@@ -53,17 +51,22 @@ or backedup (<i>mv, rsync</i> and <i>tar</i> work, <i>ssh</i> and <i>scp</i> don
 On the other hand, <i>xattr</i> comments can be bound directly to files on removable
 media (as long as the disk format allows it).
 
+<h4>database comments</h4>
+<p><i>Database</i> comments can be applied to any file, even read-only files and executables.
+
 <p>When the <i>database</i> version of a comment differs from the <i>xattr</i> version, 
 the comment box gets a light yellow background.
+
+<p>To edit a comment, first select it; to replace the comment, just type over it; to edit the comment,
+double-click the mouse, or hit &lt;Enter&gt;.
 """
 
-import sys,os,argparse,stat,getpass,shutil
-from PyQt5.QtGui import *
-from PyQt5.QtWidgets import *
-from PyQt5.QtCore import Qt, pyqtSignal
-import sqlite3, json, time
+import sys,os,time,argparse,stat,getpass,shutil,logging,math
+from   PyQt5.QtGui     import *
+from   PyQt5.QtWidgets import *
+from   PyQt5.QtCore    import Qt, pyqtSignal
+import sqlite3, json
 
-VERSION = "0.4"
 xattr_comment = "user.xdg.comment"
 xattr_author  = "user.xdg.comment.author"
 xattr_date    = "user.xdg.comment.date"
@@ -72,11 +75,18 @@ YEAR          = 3600*25*365
 
 ## globals
 mode_names = {"db":"<Database mode> ","xattr":"<Xattr mode>"} 
-modes = ("db","xattr") 
-mode = "db" 
+modes      = ("db","xattr") 
+mode       = "db" 
 
 global mainWindow, dbName
 
+verbose = None
+def print_v(*a):
+  if verbose:
+    print(*a)
+
+############# the DnDataBase, UiHelper and FileObj code is shared with other dirnotes programs
+
 DEFAULT_CONFIG_FILE = "~/.dirnotes.conf"
 
 # config
@@ -89,12 +99,28 @@ DEFAULT_CONFIG = {"xattr_tag":"user.xdg.comment",
   "options for start_mode":("db","xattr")
 }
 
-verbose = None
-def print_v(*a):
-  if verbose:
-    print(*a)
-
-class dnDataBase:
+class ConfigLoader:    # singleton
+  def __init__(self, configFile):
+    configFile = os.path.expanduser(configFile)
+    try:
+      with open(configFile,"r") as f:
+        config = json.load(f)
+    except json.JSONDecodeError:
+      errorBox(f"problem reading config file {configFile}; check the JSON syntax")
+      config = DEFAULT_CONFIG
+    except FileNotFoundError:
+      errorBox(f"config file {configFile} not found; using the default settings")
+      config = DEFAULT_CONFIG
+      try:
+        with open(configFile,"w") as f:
+          json.dump(config,f,indent=4)
+      except:
+        errorBox(f"problem creating the config file {configFile}")
+    self.dbName = os.path.expanduser(config["database"])
+    self.mode = config["start_mode"]    # can get over-ruled by the command line options
+    self.xattr_comment = config["xattr_tag"]
+
+class DnDataBase:
   ''' the database is flat
     fileName: fully qualified name
     st_mtime: a float
@@ -103,26 +129,37 @@ class dnDataBase:
     comment_time: a float, the time of the comment save
     author: the username that created the comment
 
-    the database is associated with a user, in the $HOME dir
+    this object: 1) finds or creates the database
+      2) determine if it's readonly
+
+    TODO: the database is usually associated with a user, in $XDG_DATA_HOME (~/.local/share/) 
+    TODO: if the database is not found, create it in $XDG_DATA_DIRS (/usr/local/share/)
+      make it 0666 permissions (rw-rw-rw-)
   '''
   def __init__(self,dbFile):
     '''try to open the database; if not found, create it'''
     try:
       self.db = sqlite3.connect(dbFile)
     except sqlite3.OperationalError:
-      print(f"Database {dbFile} not found")
+      logging.error(f"Database {dbFile} not found")
       raise
+ 
+    # create new database if it doesn't exist
     try:
       self.db.execute("select * from dirnotes")
     except sqlite3.OperationalError:
-      print_v("Table %s created" % ("dirnotes"))
+      print_v(f"Table dirnotes created")
       self.db.execute("create table dirnotes (name TEXT, date DATETIME, size INTEGER, comment TEXT, comment_date DATETIME, author TEXT)")
-      self.db_cursor.execute("create index dirnotes_i on dirnotes(name)") 
-  # getData is only used by the restore-from-database.......consider deleting it
-  def getData(self, fileName):
-    c = self.db.execute("select * from dirnotes where name=? and comment<>'' order by comment_date desc",(os.path.abspath(fileName),))
-    return c.fetchone()
+      self.db.execute("create index dirnotes_i on dirnotes(name)") 
+      # at this point, if a shared database is required, somebody needs to set perms to 0o666
+  
+    self.writable = True
+    try:
+      self.db.execute("pragma user_verson=0")
+    except sqlite3.OperationalError:
+      self.writable = False
 
+class UiHelper:
   @staticmethod
   def epochToDb(epoch):
     return time.strftime(DATE_FORMAT,time.localtime(epoch))
@@ -134,147 +171,194 @@ class dnDataBase:
     now = time.time()
     diff = now - longDate
     if diff > YEAR:
-      fmt = "%b %e %Y"
+      fmt = "%b %e  %Y"
     else:
-      fmt = "%b %d %H:%M"
+      fmt = "%b %e %H:%M"
     return time.strftime(fmt, time.localtime(longDate))
-    
+  @staticmethod
+  def getShortSize(fo):
+    if fo.isDir():
+      return " <DIR> "
+    elif fo.isLink():
+      return " <LINK>"
+    size = fo.getSize()
+    log = int((math.log10(size+1)-2)/3)
+    s = " KMGTE"[log]
+    base = int(size/math.pow(10,log*3))
+    return f"{base}{s}".strip().rjust(7)
+
 
 ## one for each file
 ## and a special one for ".." parent directory
-class FileObj():
-  FILE_IS_DIR    = -1
-  FILE_IS_LINK   = -2
-  FILE_IS_SOCKET = -3
-  def __init__(self, fileName):
-    self.fileName = os.path.abspath(fileName) # full path; dirs end WITHOUT a terminal /
-    self.displayName = os.path.split(fileName)[1]   # base name; dirs end with a /
-    s = os.lstat(self.fileName)
-    self.date = s.st_mtime
-    if stat.S_ISDIR(s.st_mode):
-      self.size = FileObj.FILE_IS_DIR
+class FileObj:
+  """  The FileObj knows about both kinds of comments. """
+  def __init__(self, fileName, db):
+    self.fileName = os.path.abspath(fileName)     # full path; dirs end WITHOUT a terminal /
+    self.stat = os.lstat(self.fileName)
+    self.displayName = os.path.split(fileName)[1] # base name; dirs end with a /
+    if self.isDir():
       if not self.displayName.endswith('/'):
         self.displayName += '/'
-    elif stat.S_ISLNK(s.st_mode):
-      self.size = FileObj.FILE_IS_LINK
-    elif stat.S_ISSOCK(s.st_mode):
-      self.size = FileObj.FILE_IS_SOCKET
-    else:
-      self.size = s.st_size
-    self.xattrComment = ''
-    self.xattrAuthor = None
-    self.xattrDate = None
-    self.dbComment = ''
-    self.dbAuthor = None
-    self.dbDate = None
-    self.commentsDiffer = False
-    try:
-      self.xattrComment = os.getxattr(fileName, xattr_comment, follow_symlinks=False).decode()
-      self.xattrAuthor  = os.getxattr(fileName, xattr_author, follow_symlinks=False).decode()
-      self.xattrDate    = os.getxattr(fileName, xattr_date, follow_symlinks=False).decode()
-      self.commentsDiffer = True if self.xattrComment == self.dbComment else False
-    except:  # no xattr comment
-      pass
+    self.date = self.stat.st_mtime
+    self.size = self.stat.st_size 
+    self.db = db
 
   def getName(self):
+    """ returns the absolute pathname """
     return self.fileName
   def getDisplayName(self):
+    """ returns just this basename of the file; dirs end in / """
     return self.displayName
 
-  # with an already open database cursor
-  def loadDbComment(self,db):
-    c = db.execute("select comment,author,comment_date from dirnotes where name=? and comment<>'' order by comment_date desc",(self.fileName,))
-    a = c.fetchone()
-    if a:
-      self.dbComment, self.dbAuthor, self.dbDate = a
-      self.commentsDiffer = True if self.xattrComment == self.dbComment else False
-
+  def getDbData(self):
+    """ returns (comment, author, comment_date) """
+    if not hasattr(self,'dbCommentAuthorDate'):
+      cad = self.db.execute("select comment, author, comment_date from dirnotes where name=? order by comment_date desc",(self.fileName,)).fetchone()
+      self.dbCommentAuthorDate = cad if cad else (None, None, None)
+    return self.dbCommentAuthorDate
   def getDbComment(self):
-    return self.dbComment
-  def getDbAuthor(self):
-    return self.dbAuthor
-  def getDbDate(self):
-    return self.dbDate
-  def setDbComment(self,db,newComment):
+    return self.getDbData()[0]
+
+  def getXattrData(self):
+    """ returns (comment, author, comment_date) """
+    if not hasattr(self,'xattrCommentAuthorDate'):
+      c = a = d = None
+      try:
+        c = os.getxattr(self.fileName, xattr_comment, follow_symlinks=False).decode()
+        a = os.getxattr(self.fileName, xattr_author, follow_symlinks=False).decode()
+        d = os.getxattr(self.fileName, xattr_date, follow_symlinks=False).decode()
+      except:  # no xattr comment
+        pass
+      self.xattrCommentAuthorDate = c,a,d
+    return self.xattrCommentAuthorDate
+  def getXattrComment(self):
+    return self.getXattrData()[0]
+
+  def setDbComment(self,newComment):
+    # how are we going to hook this?
+    #if not self.db.writable:
+    #  errorBox("The database is readonly; you cannot add or edit comments")
+    #  return
     s = os.lstat(self.fileName)
     try:
-      # TODO: copy from /g/test_file to /home/patb/project/dirnotes/r    fails on database.commit()
-      print_v(f"setDbComment db {db}, file: {self.fileName}")
-      print_v("insert into dirnotes (name,date,size,comment,comment_date,author) values (?,datetime(?,'unixepoch','localtime'),?,?,datetime(?,'unixepoch','localtime'),?)",
-          (self.fileName, s.st_mtime, s.st_size,
-          str(newComment), time.time(), getpass.getuser()))
-      db.execute("insert into dirnotes (name,date,size,comment,comment_date,author) values (?,datetime(?,'unixepoch','localtime'),?,?,datetime(?,'unixepoch','localtime'),?)",
+      print_v(f"setDbComment db {self.db}, file: {self.fileName}")
+      self.db.execute("insert into dirnotes (name,date,size,comment,comment_date,author) values (?,datetime(?,'unixepoch','localtime'),?,?,datetime(?,'unixepoch','localtime'),?)",
           (self.fileName, s.st_mtime, s.st_size,
           str(newComment), time.time(), getpass.getuser()))
-      print_v(f"setDbComment, execute done, about to commit()")
-      db.commit()
-      print_v(f"database write for {self.fileName}")
-      self.dbComment = newComment
+      self.db.commit()
+      self.dbCommentAuthorDate = newComment, getpass.getuser(), UiHelper.epochToDb(time.time())
     except sqlite3.OperationalError:
       print_v("database is locked or unwritable")
       errorBox("the database that stores comments is locked or unwritable")
-    self.commentsDiffer = True if self.xattrComment == self.dbComment else False
 
-  def getXattrComment(self):
-    return self.xattrComment
-  def getXattrAuthor(self):
-    return self.xattrAuthor
-  def getXattrDate(self):
-    print_v(f"someone accessed date on {self.fileName} {self.xattrDate}")
-    return self.xattrDate
   def setXattrComment(self,newComment):
     print_v(f"set comment {newComment} on file {self.fileName}")
     try:
       os.setxattr(self.fileName,xattr_comment,bytes(newComment,'utf8'),follow_symlinks=False)
       os.setxattr(self.fileName,xattr_author,bytes(getpass.getuser(),'utf8'),follow_symlinks=False)
       os.setxattr(self.fileName,xattr_date,bytes(time.strftime(DATE_FORMAT),'utf8'),follow_symlinks=False)
-      self.xattrAuthor = getpass.getuser()
-      self.xattrDate = time.strftime(DATE_FORMAT)      # alternatively, re-instantiate this FileObj
-      self.xattrComment = newComment
-      self.commentsDiffer = True if self.xattrComment == self.dbComment else False
+      self.xattrCommentAuthorDate = newComment, getpass.getuser(), time.strftime(DATE_FORMAT) 
       return True
     # we need to move these cases out to a handler 
     except Exception as e:
-      if self.size == FileObj.FILE_IS_LINK:
+      if self.isLink():
         errorBox("Linux does not allow xattr comments on symlinks; comment is stored in database")
-      elif self.size == FileObj.FILE_IS_SOCKET:
+      elif self.isSock():
         errorBox("Linux does not allow comments on sockets; comment is stored in database")
       elif os.access(self.fileName, os.W_OK)!=True:
         errorBox("you don't appear to have write permissions on this file")
         # change the listbox background to yellow
-        self.displayBox.notifyUnchanged()
       elif "Errno 95" in str(e):
         errorBox("is this a VFAT or EXFAT volume? these don't allow comments")
-      self.commentsDiffer = True
       return False
-    self.commentsDiffer = True if self.xattrComment == self.dbComment else False
 
-  def getComment(self):
-    return self.getDbComment() if mode == "db" else self.getXattrComment()
-  def getOtherComment(self):
+  def getComment(self,mode):
+    return self.getDbComment() if mode == "db"    else self.getXattrComment()
+  def getOtherComment(self,mode):
     return self.getDbComment() if mode == "xattr" else self.getXattrComment()
+  def getData(self,mode):
+    """ returns (comment, author, comment_date) """
+    return self.getDbData()    if mode == "db"    else self.getXattrData()
+  def getOtherData(self,mode):
+    """ returns (comment, author, comment_date) """
+    return self.getDbData()    if mode == "xattr" else self.getXattrData()
+
   def getDate(self):
     return self.date
   def getSize(self):
     return self.size
   def isDir(self):
-    return self.size == self.FILE_IS_DIR
+    return stat.S_ISDIR(self.stat.st_mode)
   def isLink(self):
-    return self.size == self.FILE_IS_LINK
+    return stat.S_ISLNK(self.stat.st_mode)
+  def isSock(self):
+    return stat.S_ISSOCK(self.stat.st_mode)
+
+  def copyFile(self, destDir):
+    # NOTE: this method copies the xattr (comment + old author + old date) 
+    #       but creates new db (comment + this author + new date)
+    dest = os.path.join(destDir,self.displayName)
+    try:
+      print_v("try copy from",self.fileName,"to",dest)
+      shutil.copy2(self.fileName, dest)
+    except:
+      errorBox(f"file copy to <{dest}> failed; check permissions")
+      return
+    f = FileObj(dest, self.db)
+    f.setDbComment(self.getDbComment())
+  def moveFile(self, destDir):
+    # NOTE: this method moves the xattr (comment + old author + old date)
+    #       but creates new db (comment + this author + new date)
+    src  = self.fileName
+    dest = os.path.join(destDir, self.displayName)
+    # move preserves dates & chmod/chown & xattr
+    print_v(f"move from {self.fileName} to {destDir}")
+    try:
+      shutil.move(src, dest)
+    except:
+      ErrorBox(f"file move to <{dest}> failed; check permissions")
+      return
+    # and copy the database record
+    f = FileObj(dest,self.db)
+    f.setDbComment(self.getDbComment())  
 
 class HelpWidget(QDialog):
   def __init__(self, parent):
     super(QDialog, self).__init__(parent)
-    self.layout = QVBoxLayout(self)
-    self.tb = QLabel(self)
-    self.tb.setWordWrap(True)
-    self.tb.setText(helpMsg)
-    self.tb.setFixedWidth(500)
-    self.pb = QPushButton('OK',self)
-    self.pb.setFixedWidth(200)
-    self.layout.addWidget(self.tb)
-    self.layout.addWidget(self.pb)
-    self.pb.clicked.connect(self.close)
+    layout = QVBoxLayout(self)
+
+    tb = QLabel(self)
+    tb.setWordWrap(True)
+    tb.setText(helpMsg)
+    tb.setFixedWidth(500)
+    pb = QPushButton('OK',self)
+    pb.setFixedWidth(200)
+    kb = QPushButton('Keyboard Help',self)
+
+    layout.addWidget(tb)
+    lowerBox = QHBoxLayout(self)
+    lowerBox.addWidget(pb)
+    lowerBox.addWidget(kb)
+    layout.addLayout(lowerBox)
+
+    pb.clicked.connect(self.close)
+    kb.clicked.connect(self.showKeyboardHelp)
+    self.show()
+  def showKeyboardHelp(self):
+    KeyboardHelpWidget(self)
+
+class KeyboardHelpWidget(QDialog):
+  def __init__(self, parent):
+    super(QDialog, self).__init__(parent)
+    layout = QVBoxLayout(self)
+    tb = QLabel(self)
+    tb.setWordWrap(True)
+    tb.setText(keyboardHelpMsg)
+    tb.setFixedWidth(500)
+    pb = QPushButton('OK',self)
+    layout.addWidget(tb)
+    layout.addWidget(pb)
+    pb.clicked.connect(self.close)
     self.show()
 
 class errorBox(QDialog):
@@ -292,6 +376,30 @@ class errorBox(QDialog):
     self.pb.clicked.connect(self.close)
     self.show()
 
+keyboardHelpMsg = """
+<h2>Keyboard Shortcuts</h2>
+<p>
+<table width=100%>
+<tr><td><i>Arrows</i></td><td>normal movement through the table</td></tr>
+<tr><td>Ctrl+N</td><td>sort the listing by filename</td></tr>
+<tr><td>Ctrl+D</td><td>sort the listing by date</td></tr>
+<tr><td>Ctrl+S</td><td>sort the listing by size</td></tr>
+<tr><td>Ctrl+T</td><td>sort the listing by comment</td></tr>
+<tr><td>Ctrl+M</td><td>toggle between <i>database</i> and <i>xattr</i> views</td></tr>
+<tr><td>Alt+C </td><td>copy the file <i>and its comments</i></td></tr>
+<tr><td>Alt+M </td><td>copy the file <i>and its comments</i></td></tr>
+
+<tr><td>1st column: <i>any letter</i></td><td>jump to file beginning with that letter</td></tr>
+<tr><td>1st column: &lt;Enter&gt;    </td><td>change directory</td></tr>
+<tr><td>4th column: <i>any letter</i></td><td>create a comment; replace any existing comment</td></tr>
+<tr><td>4th column: &lt;Enter&gt;    </td><td>open an existing comment for edit</td></tr>
+
+<tr><td>Ctrl+Q</td><td>quit the app</td></tr>
+</table>
+<p>
+NOTE: In edit mode, Ctrl+C, Ctrl+V and Ctrl+P work for cut, copy and paste.
+"""
+
 icon = ["32 32 6 1",  # the QPixmap constructor allows for str[]
 "   c None",
 ".  c #666666",
@@ -336,35 +444,25 @@ icon = ["32 32 6 1",  # the QPixmap constructor allows for str[]
 # sortable TableWidgetItem, based on idea by Aledsandar
 # http://stackoverflow.com/questions/12673598/python-numerical-sorting-in-qtablewidget
 # NOTE: the QTableWidgetItem has setData() and data() which may allow data bonding
+#    we could use the Qt.DisplayRole/Qt.EditRole for display, and Qt.UserRole for sorting
 # in Qt5, data() binding is more awkward, so do it here
 class SortableTableWidgetItem(QTableWidgetItem):
   def __init__(self, text, sortValue, file_object):
-    QTableWidgetItem.__init__(self, text, QTableWidgetItem.UserType)
+    QTableWidgetItem.__init__(self, text, QTableWidgetItem.UserType + 1)
     self.sortValue = sortValue
     self.file_object = file_object
   def __lt__(self, other):
     return self.sortValue < other.sortValue
     
-    
 class DirNotes(QMainWindow):
   ''' the main window of the app
     '''
-  def __init__(self, argFilename, db, start_mode, parent=None):
+  def __init__(self, argFilename, db, parent=None):
     super(DirNotes,self).__init__(parent)
     self.db = db
     self.refilling = False
     self.parent = parent
 
-    win = QWidget()
-    self.setCentralWidget(win)
-
-    lb = QTableWidget()
-    self.lb = lb
-    lb.setColumnCount(4)
-    lb.horizontalHeader().setSectionResizeMode( 3, QHeaderView.Stretch );
-    lb.verticalHeader().setDefaultSectionSize(20);  # thinner rows
-    lb.verticalHeader().setVisible(False)
-
     longPathName = os.path.abspath(argFilename)
     print_v("longpathname is {}".format(longPathName))
     if os.path.isdir(longPathName):
@@ -373,21 +471,49 @@ class DirNotes(QMainWindow):
     else:
       self.curPath, filename = os.path.split(longPathName)
     print_v("working on <"+self.curPath+"> and <"+filename+">")
+
+    win = QWidget()
+    self.setCentralWidget(win)
     
-    layout = QVBoxLayout()
+    mb = self.menuBar()
+    mf = mb.addMenu('&File')
+    mf.addAction("Sort by name", self.sbn, "Ctrl+N")
+    mf.addAction("Sort by date", self.sbd, "Ctrl+D")
+    mf.addAction("Sort by size", self.sbs, "Ctrl+Z")
+    mf.addAction("Sort by comment", self.sbc, "Ctrl+T")
+    mf.addSeparator()
+    mf.addAction("Change mode", self.switchMode, "Ctrl+M")
+    mf.addAction("Copy file", self.copyFile, "Alt+C")
+    mf.addAction("Move file", self.moveFile, "Alt+M")
+    mf.addSeparator()
+    mf.addAction("Quit", self.close, QKeySequence.Quit)
+    mf.addAction("About", self.about, QKeySequence.HelpContents)
 
-    copyIcon = QIcon.fromTheme('drive-harddisk-symbolic')
-    changeIcon = QIcon.fromTheme('emblem-synchronizing-symbolic')
+    self.setWindowTitle("==DirNotes==   Dir: "+self.curPath)
+    self.setMinimumSize(600,700)
+    self.setWindowIcon(QIcon(QPixmap(icon)))
 
-    topLayout = QHBoxLayout()
+    lb = QTableWidget()
+    self.lb = lb
+    lb.setColumnCount(4)
+    lb.horizontalHeader().setSectionResizeMode( 3, QHeaderView.Stretch );
+    lb.verticalHeader().setDefaultSectionSize(20);  # thinner rows
+    lb.verticalHeader().setVisible(False)
+    
     self.modeShow = QLabel(win)
+    copyIcon   = QIcon.fromTheme('edit-copy')
+    changeIcon = QIcon.fromTheme('emblem-synchronizing')
+    bmode = QPushButton(changeIcon, "change mode (ctrl+m)",win)
+    cf    = QPushButton(copyIcon, "copy file",win)
+    self.thisDirLabel = QLabel(win)
+
+    layout = QVBoxLayout()
+    topLayout = QHBoxLayout()
     topLayout.addWidget(self.modeShow)
-    bmode = QPushButton(changeIcon, "change mode",win)
     topLayout.addWidget(bmode)
-    cf = QPushButton(copyIcon, "copy file",win)
     topLayout.addWidget(cf)
     layout.addLayout(topLayout)
-
+    layout.addWidget(self.thisDirLabel)
     layout.addWidget(lb)
     win.setLayout(layout)
     
@@ -402,49 +528,34 @@ class DirNotes(QMainWindow):
     lb.setHorizontalHeaderItem(3,QTableWidgetItem("Comment"))
     lb.setSortingEnabled(True)
 
+    self.sameBrush   = QBrush(QColor(255,255,255))
+    self.differBrush = QBrush(QColor(255,255,160))
+
     self.refill()
     lb.resizeColumnsToContents()
     
-    if len(filename)>0:
+    if filename:
       for i in range(lb.rowCount()):
-        if filename == lb.item(i,0).data(32).getDisplayName():
-          lb.setCurrentCell(i,3)
+        if filename == lb.item(i,0).file_object.getDisplayName():
+          lb.setCurrentCell(i,0)
           break
-    
-    mb = self.menuBar()
-    mf = mb.addMenu('&File')
-    mf.addAction("Sort by name", self.sbn, "Ctrl+N")
-    mf.addAction("Sort by date", self.sbd, "Ctrl+D")
-    mf.addAction("Sort by size", self.sbs, "Ctrl+Z")
-    mf.addAction("Sort by comment", self.sbc, "Ctrl+.")
-    mf.addAction("Restore comment from database", self.restore_from_database, "Ctrl+R")
-    mf.addSeparator()
-    mf.addAction("Quit", self.close, QKeySequence.Quit)
-    mf.addAction("About", self.about, QKeySequence.HelpContents)
-
-    self.setWindowTitle("DirNotes   Alt-F for menu  Dir: "+self.curPath)
-    self.setMinimumSize(600,700)
-    self.setWindowIcon(QIcon(QPixmap(icon)))
     lb.setFocus()
 
-  def closeEvent(self,e):
-    print("closing")
   def sbd(self):
-    print("sort by date")
+    print_v("sort by date")
     self.lb.sortItems(1,Qt.DescendingOrder)
   def sbs(self):
-    print("sort by size")
+    print_v("sort by size")
     self.lb.sortItems(2)
   def sbn(self):
-    print("sort by name")
+    print_v("sort by name")
     self.lb.sortItems(0)
+  def sbc(self):
+    print_v("sort by comment")
+    self.lb.sortItems(3)
   def about(self):
     HelpWidget(self)
-  def sbc(self):
-    print("sort by comment")
-    self.lb.sortItems(3,Qt.DescendingOrder)
-  def newDir(self):
-    print("change dir to "+self.dirLeft.currentPath())
+
   def double(self,row,col):
     print_v("double click {} {}".format(row, col))
     fo = self.lb.item(row,0).file_object
@@ -452,38 +563,57 @@ class DirNotes(QMainWindow):
       print_v("double click on {}".format(fo.getName()))
       self.curPath = fo.getName()
       self.refill()
-  def copyFile(self):
+  def keyPressEvent(self,e):
+    if e.key() in (Qt.Key_Return, Qt.Key_Enter): 
+      col = self.lb.currentColumn()
+      fo = self.lb.item(self.lb.currentRow(),0).file_object
+      if col==0 and fo and fo.isDir():
+        self.curPath = fo.getName()
+        self.refill()
+        return
+      if col==3:
+        self.lb.editItem(self.lb.currentItem())
+        return
+    #self.lb.superKeyEvent(e)
+    super().keyPressEvent(e)
+
+  def copyMoveFile(self, doCopy, pickerTitle):
     # get current selection
     r, c = self.lb.currentRow(), self.lb.currentColumn()
     fo = self.lb.item(r,c).file_object
-    if not fo.isDir() and not fo.isLink():  # TODO: add check for socket 
-      print_v(f"copy file {fo.getName()}")
+    if not fo.isDir() and not fo.isLink() and not fo.isSock(): 
+      print_v(f"{'copy' if doCopy=='copy' else 'move'} file {fo.getName()}")
       # open the dir.picker
-      r = QFileDialog.getExistingDirectory(self.parent, "Select destination for FileCopy")
-      print_v(f"copy to {r}")
-      if r:
-        dest = os.path.join(r,fo.getDisplayName())
-        try:
-          shutil.copy2(fo.getName(), dest) # copy2 preserves the xattr
-          f = FileObj(dest)       # can't make the FileObj until it exists
-          f.setDbComment(self.db,fo.getDbComment())
-        except:
-          errorBox(f"file copy to <{dest}> failed; check permissions")
-    pass
-      
+      d = QFileDialog.getExistingDirectory(self.parent, pickerTitle)
+      if d:
+        print_v(f"senf file to {d}")
+        fo.copyFile(d) if doCopy=='copy' else fo.moveFile(d)
+  def copyFile(self):
+    self.copyMoveFile('copy',"Select destination for FileCopy")
+  def moveFile(self):
+    self.copyMoveFile('move',"Select destination for FileMove")
+    self.refill()
+
   def refill(self):
     self.refilling = True
     self.lb.sortingEnabled = False
+    self.directory = FileObj(self.curPath,self.db)
     
-    (self.modeShow.setText("View and edit file comments stored in extended attributes\n(xattr: user.xdg.comment)") 
+    self.thisDirLabel.setText(f'<table width=100%><tr><th><b>{self.directory.getDisplayName()}</b></th><th style"text-align:right;">{self.directory.getComment(mode)}</th></tr></table>')
+    (self.modeShow.setText("<i>Showing comments stored in extended attributes</i><br>(xattr: user.xdg.comment)") 
       if mode=="xattr" else 
-      self.modeShow.setText("View and edit file comments stored in the database \n(~/.dirnotes.db)"))
+      self.modeShow.setText("<i>Showing comments from the database</i><br>(~/.dirnotes.db)"))
     self.lb.clearContents()
-    small_font = QFont("",8)
-    dirIcon = QIcon.fromTheme('folder')
+    dirIcon  = QIcon.fromTheme('folder')
     fileIcon = QIcon.fromTheme('text-x-generic')
     linkIcon = QIcon.fromTheme('emblem-symbolic-link')
-    current, dirs, files = next(os.walk(self.curPath,followlinks=True))
+    sockIcon = QIcon.fromTheme('emblem-shared')
+
+    try:
+      current, dirs, files = next(os.walk(self.curPath,followlinks=True))
+    except:
+      print(f"{self.curPath} is not a valid directory")
+      sys.exit(1)
     dirs.sort()
     files.sort()
     
@@ -492,57 +622,57 @@ class DirNotes(QMainWindow):
     d = dirs + files
     self.lb.setRowCount(len(d))
 
-    #~ self.files = {}
-    self.files = []
-    # this is a list of all the file
     #~ print("insert {} items into cleared table {}".format(len(d),current))
     for i,name in enumerate(d):
-      this_file = FileObj(os.path.join(current,name))
-      this_file.loadDbComment(self.db)
-      print_v("FileObj created as {} and the db-comment is <{}>".format(this_file.displayName, this_file.dbComment))
+      this_file = FileObj(os.path.join(current,name),self.db)
+      print_v("FileObj created as {} and the db-comment is <{}>".format(this_file.displayName, this_file.getDbComment))
       #~ print("insert order check: {} {} {} {}".format(d[i],i,this_file.getName(),this_file.getDate()))
-      #~ self.files.update({this_file.getName(),this_file})
-      self.files = self.files + [this_file]
       display_name = this_file.getDisplayName()
-      if this_file.getSize() == FileObj.FILE_IS_DIR:
+      if this_file.isDir():
         item = SortableTableWidgetItem(display_name,' '+display_name, this_file)  # directories sort first
       else:
         item = SortableTableWidgetItem(display_name,display_name, this_file)
-      item.setData(32,this_file)  # keep a hidden copy of the file object
       item.setToolTip(this_file.getName())
+      item.setFlags(Qt.ItemIsEnabled)
       self.lb.setItem(i,0,item)
-      #lb.itemAt(i,0).setFlags(Qt.ItemIsEnabled) #NoItemFlags) 
 
       # get the comment from database & xattrs, either can fail
-      comment = this_file.getComment()
-      other_comment = this_file.getOtherComment()
-      ci = SortableTableWidgetItem(comment,'',this_file)
-      ci.setToolTip(f"comment: {comment}\ncomment date: {this_file.getDbDate()}\nauthor: {this_file.getDbAuthor()}")
+      comment, auth, cdate = this_file.getData(mode)
+      other_comment = this_file.getOtherComment(mode)
+      ci = SortableTableWidgetItem(comment,comment or '~',this_file)
+      ci.setToolTip(f"comment: {comment}\ncomment date: {cdate}\nauthor: {auth}")
       if other_comment != comment:
-        ci.setBackground(QBrush(QColor(255,255,160)))
+        ci.setBackground(self.differBrush)
         print_v("got differing comments <{}> and <{}>".format(comment, other_comment))
+      ci.setFlags(Qt.ItemIsSelectable | Qt.ItemIsEditable | Qt.ItemIsEnabled)
       self.lb.setItem(i,3,ci)
 
       dt = this_file.getDate()
-      da = SortableTableWidgetItem(dnDataBase.getShortDate(dt),dt,this_file)
+      da = SortableTableWidgetItem(UiHelper.getShortDate(dt),dt,this_file)
       da.setToolTip(time.strftime(DATE_FORMAT,time.localtime(dt)))
+      da.setFlags(Qt.ItemIsEnabled)
       self.lb.setItem(i,1,da)
 
       si = this_file.getSize()
       if this_file.isDir():
-        sa = SortableTableWidgetItem('',0,this_file)
-        item.setIcon(dirIcon)
+        sa = SortableTableWidgetItem(UiHelper.getShortSize(this_file),0,this_file)
+        sa.setIcon(dirIcon)
       elif this_file.isLink():
-        sa = SortableTableWidgetItem('symlink',-1,this_file)
-        item.setIcon(linkIcon)
+        sa = SortableTableWidgetItem(UiHelper.getShortSize(this_file),-1,this_file)
+        sa.setIcon(linkIcon)
         dst = os.path.realpath(this_file.getName())
         sa.setToolTip(f"symlink: {dst}")
+      elif this_file.isSock():
+        sa = SortableTableWidgetItem(UiHelper.getShortSize(this_file),-1,this_file)
+        sa.setIcon(sockIcon)
       else:
-        sa = SortableTableWidgetItem(str(si),si,this_file)
-        item.setIcon(fileIcon)
+        sa = SortableTableWidgetItem(UiHelper.getShortSize(this_file),si,this_file)
+        sa.setIcon(fileIcon)
       sa.setTextAlignment(Qt.AlignRight)
+      sa.setFlags(Qt.ItemIsEnabled)
       self.lb.setItem(i,2,sa)
 
+    self.lb.setCurrentCell(0,0)
     self.refilling = False
     self.lb.sortingEnabled = True
     self.lb.resizeColumnToContents(1)
@@ -550,41 +680,28 @@ class DirNotes(QMainWindow):
   def change(self,x):
     if self.refilling:
       return
-    print_v("debugging " + x.text() + " r:" + str(x.row()) + " c:" + str(x.column()))
-    print_v("      selected file: "+self.lb.item(x.row(),0).file_object.getName())
     the_file = self.lb.item(x.row(),0).file_object
-    print_v("      and the row file is "+the_file.getName())
-    the_file.setDbComment(self.db,str(x.text()))
-    r = the_file.setXattrComment(str(x.text())) 
-    if r:
-      the_file.setDbComment(self.db,x.text())
+    print_v(f"debugging {x.text()} r:{str(x.row())} c:{str(x.column())}")
+    print_v(f"      selected file: {the_file.getName()}")
+    the_file.setDbComment(str(x.text()))
+    the_file.setXattrComment(str(x.text())) 
+
+    # set the background (wrap it, because of reentry to .change())
+    self.refilling = True 
+    if the_file.getComment(mode) == the_file.getOtherComment(mode):
+      x.setBackground(self.sameBrush)
+    else:
+      x.setBackground(self.differBrush)
+    self.refilling = False
 
   def switchMode(self):
     global mode
     mode = "xattr" if mode == "db" else "db" 
+    row,column = self.lb.currentRow(), self.lb.currentColumn()
     self.refill()
+    self.lb.setCurrentCell(row,column)
+    self.lb.setFocus(True)
 
-  # TODO: this may not be needed
-  def restore_from_database(self):
-    print("restore from database")
-    # retrieve the full path name
-    fileName = str(self.lb.item(self.lb.currentRow(),0).file_object.getName())
-    print("using filename: "+fileName)
-    existing_comment = str(self.lb.item(self.lb.currentRow(),3).text())
-    print("restore....existing="+existing_comment+"=")
-    if len(existing_comment) > 0:
-      m = QMessageBox() 
-      m.setText("This file already has a comment. Overwrite?")
-      m.setStandardButtons(QMessageBox.Ok | QMessageBox.Cancel);
-      if m.exec_() != QMessageBox.Ok:
-        return
-    fo_row = self.db.getData(fileName)
-    if fo_row and len(fo_row)>1:
-      comment = fo_row[3]
-      print(fileName,fo_row[0],comment) 
-      the_file = dn.files[self.lb.currentRow()]
-      the_file.setComment(comment)
-      self.lb.setItem(self.lb.currentRow(),3,QTableWidgetItem(comment))
     
 def parse():
   parser = argparse.ArgumentParser(description='dirnotes application')
@@ -614,33 +731,28 @@ if __name__=="__main__":
   print_v(f"using {p.dirname}")
   verbose = p.verbose
   
-  config_file = p.config_file if p.config_file else DEFAULT_CONFIG_FILE
-  config_file = os.path.expanduser(config_file)
-  config = DEFAULT_CONFIG
-  try:
-    with open(config_file,"r") as f:
-      config = json.load(f)
-  except json.JSONDecodeError:
-    print(f"problem reading config file {config_file}; check the .json syntax")
-  except FileNotFoundError:
-    print(f"config file {config_file} not found, using the default settings and writing a default")
-    try:
-      with open(config_file,"w") as f:
-        json.dump(config,f,indent=4)
-    except:
-      print(f"problem creating the file {config_file}")
+  config = ConfigLoader(p.config_file or DEFAULT_CONFIG_FILE)
   
   print_v(f"here is the .json {repr(config)}")
-  dbName = os.path.expanduser(config["database"])
-  db = dnDataBase(dbName).db
-  xattr_comment = config["xattr_tag"]
+  dbName = config.dbName 
+  db = DnDataBase(dbName).db
+  xattr_comment = config.xattr_comment
   xattr_author  = xattr_comment + ".author"
   xattr_date    = xattr_comment + ".date"
 
-  mode = "xattr" if p.xattr else "db"
+  mode = config.mode 
+  if p.xattr:
+    mode = "xattr" 
+  if p.db:
+    mode = "db"
 
   a = QApplication([])
-  mainWindow = DirNotes(p.dirname,db,config["start_mode"])
+  # TODO: add 'mode' as an argument to contructor; add setMode() as a method
+  mainWindow = DirNotes(p.dirname,db)
+  if p.sort_by_size:
+    mainWindow.sbs()
+  if p.sort_by_date:
+    mainWindow.sbd()
   mainWindow.show()
   
   if p.sort_by_date:
@@ -650,21 +762,6 @@ if __name__=="__main__":
 
   a.exec_()
   
-  #xattr.setxattr(filename,COMMENT_KEY,commentText)
-
-''' files from directories
-use os.isfile()
-os.isdir()
-current, dirs, files = os.walk("path").next()
-possible set folllowLinks=True'''
-
-''' notes from the wdrm project
-table showed 
-filename, size, date size, date, desc
-
-at start, fills the list of all the files
-skip the . entry
-'''
 
 ''' should we also do user.xdg.tags="TagA,TagB" ?
 user.charset
@@ -674,19 +771,19 @@ user.xdg.language=[RFC3066/ISO639]
 user.xdg.publisher
 '''
 
-''' TODO: add cut-copy-paste for comments '''
-
 ''' TODO: also need a way to display-&-restore comments from the database '''
 
 ''' TODO: implement startup -s and -m for size and date '''
 
-''' TODO: add an icon for the app '''
-
 ''' TODO: create 'show comment history' popup '''
-
-''' TODO: add dual-pane for file-move, file-copy '''
   
 ''' commandline xattr
 getfattr -h (don't follow symlink) -d (dump all properties)
 '''
-''' if the args line contains a file, jump to it '''
+
+''' CODING NOTES:
+  in FileObj, the long absolute name always ends without a /
+    the short display name ends with a / if it's a directory
+  dates are always in YYYY-MM-DD HH:MM:SS format
+    these can be sorted
+'''

dirnotes-cli

@@ -1,120 +1,372 @@
 #!/usr/bin/python3
-# TODO starting with a dir shoild list all-files
-# TODO: get rid of the -n option; multiple files are prefixed by 'filename:', single files aren't
+# TODO: option to print out full path name; most useful in the .json output format
 
-VERSION = "0.3"
+VERSION = "0.4"
 
 import os, sys, argparse, xattr, json, sqlite3
 
+# global mutables 
 answer_json = []
-verbose = 0
+verbose = debug = 0
 db = None
 xattr_comment = "user.xdg.comment"
-xattr_author    = "user.xdg.comment.author"
-xattr_date      = "user.xdg.comment.date"
+xattr_author  = "user.xdg.comment.author"
+xattr_date    = "user.xdg.comment.date"
+mode          = "db"
 
 #======= debugging/verbose ===========
 def print_d(*a):
-    if verbose > 1:
-        print('>>',*a)
-def print_v(*a):
-    if verbose:
-        print('>',*a)
+    if debug:
+        print('>>', *a)
+def errorBox(*a):
+    print(*a)
+
+# >>> snip here <<<
+#============ the DnDataBase, UiHelper and FileObj code is shared with other dirnotes programs
+import getpass, time, stat, shutil
+
+DEFAULT_CONFIG_FILE = "~/.config/dirnotes/dirnotes.conf" # or /etc/dirnotes.conf
+
+# config
+#    we could store the config in the database, in a second table
+#    or in a .json file
+DEFAULT_CONFIG = {"xattr_tag":"user.xdg.comment",
+  "database":"~/.local/share/dirnotes/dirnotes.db",
+  "start_mode":"xattr",
+  "options for database":("~/.local/share/dirnotes/dirnotes.db","~/.dirnotes.db","/etc/dirnotes.db"),
+  "options for start_mode":("db","xattr")
+}
+
+class ConfigLoader:    # singleton
+  def __init__(self, configFile):
+    configFile = os.path.expanduser(configFile)
+    try:
+      with open(configFile,"r") as f:
+        config = json.load(f)
+    except json.JSONDecodeError:
+      errorBox(f"problem reading config file {configFile}; check the JSON syntax")
+      config = DEFAULT_CONFIG
+    except FileNotFoundError:
+      errorBox(f"config file {configFile} not found; using the default settings")
+      config = DEFAULT_CONFIG
+      try:
+        os.makedirs(os.path.dirname(configFile),exist_ok = True)
+        with open(configFile,"w") as f:
+          json.dump(config,f,indent=4)
+      except:
+        errorBox(f"problem creating the config file {configFile}")
+    self.dbName = os.path.expanduser(config["database"])
+    self.mode = config["start_mode"]    # can get over-ruled by the command line options
+    self.xattr_comment = config["xattr_tag"]
+
+class DnDataBase:
+  ''' the database is flat
+    fileName: fully qualified name
+    st_mtime: a float
+    size: a long
+    comment: a string
+    comment_time: a float, the time of the comment save
+    author: the username that created the comment
+
+    this object: 1) finds or creates the database
+      2) determine if it's readonly
+
+    TODO: the database is usually associated with a user, in $XDG_DATA_HOME (~/.local/share/)
+    TODO: if the database is not found, create it in $XDG_DATA_DIRS (/usr/local/share/)
+      make it 0666 permissions (rw-rw-rw-)
+  '''
+  def __init__(self,dbFile):
+    '''try to open the database; if not found, create it'''
+    try:
+      self.db = sqlite3.connect(dbFile)
+    except sqlite3.OperationalError:
+      print_d(f"Database {dbFile} not found")
+      try: 
+        os.makedirs(os.path.dirname(dbFile), exist_ok = True)
+        self.db = sqlite3.connect(dbFile)
+      except (sqlite3.OperationalError, PermissionError):
+        printd(f"Failed to create {dbFile}, aborting")
+        raise
+
+    # create new table if it doesn't exist
+    try:
+      self.db.execute("select * from dirnotes")
+    except sqlite3.OperationalError:
+      self.db.execute("create table dirnotes (name TEXT, date DATETIME, size INTEGER, comment TEXT, comment_date DATETIME, author TEXT)")
+      self.db.execute("create index dirnotes_i on dirnotes(name)") 
+      print_d(f"Table dirnotes created")
+      # at this point, if a shared database is required, somebody needs to set perms to 0o666
+  
+    self.writable = True
+    try:
+      self.db.execute("pragma user_verson=0")
+    except sqlite3.OperationalError:
+      self.writable = False
+
+DATE_FORMAT   = "%Y-%m-%d %H:%M:%S"
+class UiHelper:
+  @staticmethod
+  def epochToDb(epoch):
+    return time.strftime(DATE_FORMAT,time.localtime(epoch))
+  @staticmethod
+  def DbToEpoch(dbTime):
+    return time.mktime(time.strptime(dbTime,DATE_FORMAT))
+  @staticmethod
+  def getShortDate(longDate):
+    now = time.time()
+    diff = now - longDate
+    if diff > YEAR:
+      fmt = "%b %e  %Y"
+    else:
+      fmt = "%b %e %H:%M"
+    return time.strftime(fmt, time.localtime(longDate))
+  @staticmethod
+  def getShortSize(fo):
+    if fo.isDir():
+      return " <DIR> "
+    elif fo.isLink():
+      return " <LINK>"
+    size = fo.getSize()
+    log = int((math.log10(size+1)-2)/3)
+    s = " KMGTE"[log]
+    base = int(size/math.pow(10,log*3))
+    return f"{base}{s}".strip().rjust(7)
+
+
+## one for each file
+## and a special one for ".." parent directory
+class FileObj:
+  """  The FileObj knows about both kinds of comments. """
+  def __init__(self, fileName, db):
+    self.fileName = os.path.abspath(fileName)     # full path; dirs end WITHOUT a terminal /
+    self.stat = os.lstat(self.fileName)
+    self.displayName = os.path.split(fileName)[1] # base name; dirs end with a /
+    if self.isDir():
+      if not self.displayName.endswith('/'):
+        self.displayName += '/'
+    self.date = self.stat.st_mtime
+    self.size = self.stat.st_size 
+    self.db = db
+
+  def getName(self):
+    """ returns the absolute pathname """
+    return self.fileName
+  def getDisplayName(self):
+    """ returns just the basename of the file; dirs end in / """
+    return self.displayName
+
+  def getDbData(self):
+    """ returns (comment, author, comment_date) """
+    if not hasattr(self,'dbCommentAuthorDate'):
+      cad = self.db.execute("select comment, author, comment_date from dirnotes where name=? order by comment_date desc",(self.fileName,)).fetchone()
+      self.dbCommentAuthorDate = cad if cad else (None, None, None)
+    return self.dbCommentAuthorDate
+  def getDbComment(self):
+    return self.getDbData()[0]
+
+  def getXattrData(self):
+    """ returns (comment, author, comment_date) """
+    if not hasattr(self,'xattrCommentAuthorDate'):
+      c = a = d = None
+      try:
+        c = os.getxattr(self.fileName, xattr_comment, follow_symlinks=False).decode()
+        a = os.getxattr(self.fileName, xattr_author, follow_symlinks=False).decode()
+        d = os.getxattr(self.fileName, xattr_date, follow_symlinks=False).decode()
+      except:  # no xattr comment
+        pass
+      self.xattrCommentAuthorDate = c,a,d
+    return self.xattrCommentAuthorDate
+  def getXattrComment(self):
+    return self.getXattrData()[0]
+
+  def setDbComment(self,newComment):
+    # how are we going to hook this?
+    #if not self.db.writable:
+    #  errorBox("The database is readonly; you cannot add or edit comments")
+    #  return
+    s = os.lstat(self.fileName)
+    try:
+      print_d(f"setDbComment db {self.db}, file: {self.fileName}")
+      self.db.execute("insert into dirnotes (name,date,size,comment,comment_date,author) values (?,datetime(?,'unixepoch','localtime'),?,?,datetime(?,'unixepoch','localtime'),?)",
+          (self.fileName, s.st_mtime, s.st_size,
+          str(newComment), time.time(), getpass.getuser()))
+      self.db.commit()
+      self.dbCommentAuthorDate = newComment, getpass.getuser(), UiHelper.epochToDb(time.time())
+    except sqlite3.OperationalError:
+      print_d("database is locked or unwritable")
+      errorBox("the database that stores comments is locked or unwritable")
+
+  def setXattrComment(self,newComment):
+    print_d(f"set comment {newComment} on file {self.fileName}")
+    try:
+      os.setxattr(self.fileName,xattr_comment,bytes(newComment,'utf8'),follow_symlinks=False)
+      os.setxattr(self.fileName,xattr_author,bytes(getpass.getuser(),'utf8'),follow_symlinks=False)
+      os.setxattr(self.fileName,xattr_date,bytes(time.strftime(DATE_FORMAT),'utf8'),follow_symlinks=False)
+      self.xattrCommentAuthorDate = newComment, getpass.getuser(), time.strftime(DATE_FORMAT) 
+      return True
+    # we need to move these cases out to a handler 
+    except Exception as e:
+      if self.isLink():
+        errorBox("Linux does not allow xattr comments on symlinks; comment is stored in database")
+      elif self.isSock():
+        errorBox("Linux does not allow comments on sockets; comment is stored in database")
+      elif os.access(self.fileName, os.W_OK)!=True:
+        errorBox(f"you don't appear to have write permissions on this file: {self.fileName}")
+        # change the listbox background to yellow
+      elif "Errno 95" in str(e):
+        errorBox("is this a VFAT or EXFAT volume? these don't allow comments")
+      return False
+
+  def getComment(self,mode):
+    """ returns the comment for the given mode """
+    return self.getDbComment() if mode == "db"    else self.getXattrComment()
+  def getOtherComment(self,mode):
+    return self.getDbComment() if mode == "xattr" else self.getXattrComment()
+  def getData(self,mode):
+    """ returns (comment, author, comment_date) for the given mode """
+    return self.getDbData()    if mode == "db"    else self.getXattrData()
+  def getOtherData(self,mode):
+    """ returns (comment, author, comment_date) for the 'other' mode """
+    return self.getDbData()    if mode == "xattr" else self.getXattrData()
+
+  def getDate(self):
+    return self.date
+  def getSize(self):
+    return self.size
+  def isDir(self):
+    return stat.S_ISDIR(self.stat.st_mode)
+  def isLink(self):
+    return stat.S_ISLNK(self.stat.st_mode)
+  def isSock(self):
+    return stat.S_ISSOCK(self.stat.st_mode)
+
+  def copyFile(self, dest, doMove = False):
+    """ dest is either a FQ filename or a FQ directory, to be expanded with same basename """
+    # NOTE: this method copies the xattr (comment + old author + old date) 
+    #       but creates new db (comment + this author + new date)
+    if os.path.isdir(dest):
+      dest = os.path.join(destDir,self.displayName)
+    try:
+      print_d("try copy from",self.fileName,"to",dest)
+      # shutil methods preserve dates & chmod/chown & xattr
+      if doMove:
+        shutil.move(self.fileName, dest)
+      else:
+        shutil.copy2(self.fileName, dest)  
+      # can raise FileNotFoundError, Permission Error, shutil.SameFileError, IsADirectoryError 
+    except:
+      errorBox(f"file copy/move to <{dest}> failed; check permissions")
+      return
+    # and copy the database record
+    f = FileObj(dest, self.db)
+    f.setDbComment(self.getDbComment())
+  def moveFile(self, dest):
+    """ dest is either a FQ filename or a FQ directory, to be expanded with same basename """
+    self.copyFile(dest, doMove = True)
+
+# >>> snip here <<<
 
 #============= the functions that are called from the main.loop ===============
 
-def file_copy(f,target,target_is_dir,force):
-    print_d(f"call file_copy with args={target},{target_is_dir} and {force}")
-    dest = target if not target_is_dir else target+'/'+os.path.basename(f)
+def file_copy(f,target,target_is_dir,is_copy,force):
+    print_d(f"call file_copy/move with args={target},{target_is_dir} and {force}")
+    dest = target if not target_is_dir else os.path.join(target,f.getDisplayName())
     if os.path.exists(dest) and not force:
-        go = input("The copy target <<" + dest + ">> exists. Overwrite? (y or n) ")
+        go = input("The copy/move target <<" + dest + ">> exists. Overwrite? (y or n) ")
         if go != 'y' and go != 'Y':
             return
-    print_d(f"copy from {f} to {dest}")
+    print_d(f"copy/move from {f} to {dest}")
+    if is_copy:
+      f.copyFile(dest)
+    else:
+      f.moveFile(dest)
 
 def file_zap(f,all_flag):
-    print_d(f"zapping the comment history of {f}")
+    db = f.db
+    print_d(f"zapping the comment history of {f.getName()}")
     if all_flag:
-        print_d("zapping the entire database")
+        confirm = input("You requested a complete flush of the comment database history. Please hit 'Y' to confirm")
+        if confirm == 'Y':
+            print_d("zapping the entire database")
+            db.execute("delete from dirnotes where comment_date < (select max(comment_date) from dirnotes d2 where d2.name = dirnotes.name)")
+    else:
+        db.execute("delete from dirnotes where name=? and comment_date < (select max(comment_date) from dirnotes where name=?)",(f.getName(),f.getName()))
+    db.commit()
 
 def file_modify_comment(f, create, append, erase):
     print_d(f"modify the comment on file {f} with extra={(create,append,erase)}")
-    if not os.path.exists(f):
+    if not os.path.exists(f.getName()):
         print(f"the target file does not exist; please check the spelling of the file: {f}")
-        # sys.exit() here?
+        sys.exit(10)
+    if create:
+        f.setXattrComment(create)
+        f.setDbComment(create)
+    elif append:
+        c = f.getComment(mode)
+        f.setXattrComment(f"{c}; {append}")
+        f.setDbComment(f"{c}; {append}")
+    elif erase:
+        f.setXattrComment('')
+        f.setDbComment('')
 
-def file_display(f, listall, history, json, minimal):
-    print_d(f"list file details {f}")
-    x_comment = None
-    try:
-        x_comment = xattr.getxattr(f,xattr_comment).decode()
-        x_author    = xattr.getxattr(f,xattr_author).decode()
-        x_date      = xattr.getxattr(f,xattr_date).decode()
-    except:
-        pass
-    full_f = os.path.realpath(f)
-
-    d_comment = getDbComment(full_f)
-    if d_comment:
-        d_comment, d_author, d_date = d_comment
-    print_d(f"for file {f}, database comment is <{d_comment}>, xattr comment is <{x_comment}>")
-
-    if os.path.isdir(f):
-        f = f+'/'
-    if x_comment or listall:
-        if x_comment and (d_comment != x_comment):
-            x_comment += '*'
+def file_display(f, listall, json, minimal):
+    fn = f.getDisplayName()
+    print_d(f"list file details {fn}")
+    c,a,d = f.getData(mode)
+    c1,a1,d1 = f.getOtherData(mode)
+
+    if c or listall:
+        if c and (c != c1):
+            c += '*'
         if not json:
             if minimal:
-                print(f"{x_comment}")
+                print(f"{c}")
+            elif verbose:
+                print(f"{f.getName()}: {repr(c)}, {repr(a)}, {repr(d)}")
             else:
-                print(f"{f}: {x_comment}")
+                print(f"{fn}: {repr(c)}")
         else:
             if verbose:
-                answer_json.append( {"file":f,"comment":x_comment,"author":x_author,"date":x_date } )
+                answer_json.append( {"file":f.getName(),"comment":c,"author":a,"date":d } )
             else:
-                answer_json.append( {"file":f,"comment":x_comment} )
-
-def getDbComment(full_filename):
-    global db
-    print_d(f"db access for {full_filename}")
-    c = db.cursor()
-    c.execute("select comment,author,comment_date from dirnotes where name=? and comment<>'' order by comment_date desc",(full_filename,))
-    a = c.fetchone()
-    if a:
-        return a[0:3]
-
-def openDb():
-    global db
-    dbName = "~/.dirnotes.db"
-    dbName = os.path.expanduser(dbName)
-    db = sqlite3.connect(dbName)
-    try:
-        c = db.cursor()
-        c.execute("select * from dirnotes")
-    except sqlite3.OperationalError:
-        c.execute("create table dirnotes (name TEXT, date DATETIME, size INTEGER, comment TEXT, comment_date DATETIME, author TEXT)")
-    return db 
+                answer_json.append( {"file":fn,"comment":c} )
+
+def file_history(f,json):
+    db = f.db
+    c = db.execute("select comment, author, comment_date from dirnotes where name=? order by comment_date desc",(f.getName(),))
+    if not json:
+        print(f"file: \t\t{f.getName()}\n")
+    else:
+        answer_json.append ( {"file":f.getName()} )
+    for a in c.fetchall():
+        if not json:
+            print(f"comment: \t{a[0]}\nauthor: \t{a[1]}\t\tdate: \t\t{a[2]}\n")
+        else:
+            answer_json.append( {"comment":a[0],"author":a[1],"date":a[2]} )
+
 
 def main(args):
     parser = argparse.ArgumentParser(description="Display or add comments to files",
         epilog="Some options conflict. Use only one of: -l -c -a -H -e -z -Z and one of -d -x")
-    parser.add_argument('-V',"--version", action="version",     version=f"dncli ver:{VERSION}")
-    parser.add_argument('-v',"--verbose", action='count',           help="verbose, almost debugging; do not use in scripts",default=0)
-    parser.add_argument('-j',"--json",      action="store_true",help="output in JSON format")
+    parser.add_argument('-V',"--version", action="version",   version=f"dncli ver:{VERSION}")
+    parser.add_argument('-v',"--verbose", action='count',     help="verbose output (include comment author & date)",default=0)
+    parser.add_argument('-D',"--debug",   action='store_true',help="include debugging output; do not use in scripts",default=0)
+    parser.add_argument('-j',"--json",    action="store_true",help="output in JSON format")
     pars_m = parser.add_mutually_exclusive_group()
     pars_m.add_argument('-l',"--listall", action="store_true",help="list all files, including those without comments")
-    parser.add_argument('-d',"--db",            action="store_true",help="list comments from database")
-    parser.add_argument('-x',"--xattr",     action="store_true",help="list comments from xattr")
+    parser.add_argument('-d',"--db",      action="store_true",help="list comments from database")
+    parser.add_argument('-x',"--xattr",   action="store_true",help="list comments from xattr")
     parser.add_argument('-n',"--minimal", action="store_true",help="output only comments; useful in scripting")
-    parser.add_argument('-H',"--history", action="store_true",help="output the history of comments for a file")
-    pars_m.add_argument('-c',"--create",    metavar="comment",  help="add a comment to a file")
-    pars_m.add_argument('-a',"--append",    metavar="comment",  help="append to a comment on a file, separator=';'")
-    pars_m.add_argument('-C',"--copy",      action="store_true",help="copy a file with its comments")
+    parser.add_argument('-H',"--history", action="store_true",help="output the history of database comments for a file")
+    pars_m.add_argument('-c',"--create",  metavar="comment",  help="add a comment to a file")
+    pars_m.add_argument('-a',"--append",  metavar="comment",  help="append to a comment on a file, separator=';'")
+    pars_m.add_argument('-C',"--copy",    action="store_true",help="copy a file with its comments")
+    pars_m.add_argument('-M',"--move",    action="store_true",help="move a file with its comments")
     parser.add_argument('-y',"--cp_force",action="store_true",help="copy over existing files")
-    pars_m.add_argument('-e',"--erase",     action="store_true",help="erase the comment on a file")
-    pars_m.add_argument('-z',"--zap",           action="store_true",help="clear the comment history on a file")
-    pars_m.add_argument('-Z',"--zapall",     action="store_true",help="clear the comment history in the entire database")
+    pars_m.add_argument('-e',"--erase",   action="store_true",help="erase the comment on a file")
+    pars_m.add_argument('-z',"--zap",     action="store_true",help="clear the database comment history on a file")
+    pars_m.add_argument('-Z',"--zapall",  action="store_true",help="clear the comment history in the entire database")
+    parser.add_argument(     "--config",  dest="config_file", help="use config file (default ~/.config/dirnotes/dirnotes.conf)")
     parser.add_argument('file_list',nargs='*',help="file(s); list commands may omit this")
     args = parser.parse_args()
 
@@ -140,9 +392,16 @@ def main(args):
         print("the -x/--xattr option doesn't apply to the -z/--zap and -Z/--zapall commands")
         sys.exit(7)
 
-    global verbose
+    global verbose, debug
     verbose = args.verbose
+    debug = args.debug
     
+    config = ConfigLoader(args.config_file or DEFAULT_CONFIG_FILE)
+    global mode
+    mode = config.mode
+    mode = "xattr" if args.xattr else ("db" if args.db else mode)
+    db = DnDataBase(config.dbName).db
+
     #====== 1) build the file list =============
 
     files = args.file_list
@@ -150,7 +409,7 @@ def main(args):
     if not files and args.display:
         files = os.listdir(".")
         files.sort()
-    # other command require explicity file lists, but 'dncli -c "new comment" *' will work
+    # other command require explicity file lists
     if not files:
         print("please specify a file or files to use")
         sys.exit(10)
@@ -166,36 +425,36 @@ def main(args):
         func = file_zap
         loop_args = (args.zapall,)
         if args.zapall:
-            files = (1,)
-    elif args.copy:
-        print_d(f"got a copy command to {args.copy}")
+            files = ('.',)
+    elif args.copy or args.move:
+        print_d(f"got a copy/move command to copy={args.copy}, move={args.move}")
         # the last item on the file list is the target
         n_files = len(files)
         if n_files < 2:
-            print_d(f"the copy command requires at least two arguments, the last one is the destination")
+            print("the copy/move command requires at least two arguments, the last one is the destination")
             sys.exit(1)
-        target = files[-1]
+        files, target = files[:-1], files[-1]
         target_is_directory = os.path.isdir(target)
-        files = files[:-1]
-        print_d(f"copy from {files} to {target}")
+        print_d(f"copy/move from {files} to {target}")
         if n_files > 2 and not target_is_directory:
-            print_d(f"multiple copy files must go to a target directory")
+            print("multiple copy/move files must go to a target directory")
             sys.exit(3)
         func = file_copy
-        loop_args = (target, target_is_directory, args.cp_force)
+        loop_args = (target, target_is_directory, args.copy, args.cp_force)
+    elif args.history:
+        func = file_history
+        loop_args = (args.json,)
     else:
-        assert(args.display)
-        print_v(f"list files using {'database' if args.db else 'xattr'} priority")
+        assert args.display
+        print_d(f"list files using {mode} priority")
         print_d(f"display command with option: {args.listall} and {args.history} and {args.json} and {args.minimal}")
-        loop_args = (args.listall, args.history, args.json, args.minimal)
+        loop_args = (args.listall, args.json, args.minimal)
         func = file_display
 
     #====== 3) loop on the list, execute the function =============
-    global db
-    db = openDb()
 
     for f in files:
-        func(f,*loop_args)
+        func(FileObj(f,db),*loop_args)
 
     if answer_json:
         print(json.dumps(answer_json))

dirnotes-cli.md

@@ -26,8 +26,8 @@ dirnotes-cli - view and manage Dirnotes file annotations
     **dirnotes-cli** -e filename...
 
   FileCopy:  
-    **dirnotes-cli** -C src-file... <dest-file | dest-dir>  
-    **dirnotes-cli** -M src-file... <dest-file | dest-dir>
+    **dirnotes-cli** -C [-y] src-file... <dest-file | dest-dir>  
+    **dirnotes-cli** -M [-y] src-file... <dest-file | dest-dir>
 
   Cleanup:  
     **dirnotes-cli** -z [filename]...   
@@ -41,31 +41,39 @@ a file. The descriptions are stored in two places:
   * in the xattr properties of the file
   * in a database located in the user's home directory
 
+The **dirnotes-cli** is designed for use on the command line, or in
+scripting.
 
 The *list* commands will display the comment from the database or xattr (as
-determined by the config file, below, or as specified with **-d**/**-x** options). If the database and xattr comments differ, the
-comment will be terminated by a '\*' character. The _-H_ option displays the history 
-of comments from the database.
+determined by the config file or as specified with **-d**/**-x** options). 
+If the database and xattr comments differ, the
+comment will be terminated by a '\*' character. 
+The **-H** option displays the history of comments from the database.
 
-The output of the *list* commands can be in .json format (_-j_) , and can optionally 
-display the comment creator and the comment date (_-v_)
+The output of the *list* commands can be in .json format (**-j**) , 
+and can optionally display the comment creator and the comment date (**-v**)
 
-The *create* commands will attempt to store the file comments in both the xattr of the file,
-and in the database. If either of these fail, they fail silently. Use the _-c_ to create a comment, use _-a_ to append to an existing comment, and _-e_ to erase a comment.
+The *create* commands will attempt to store the file comments in _both_ 
+the xattr of the file, and in the database. 
+If either of these fail, they fail silently. Use the **-c** to create a comment, 
+use **-a** to append to an existing comment, and **-e** to erase a comment.
 
-The *filecopy* commands will copy the file to a destination, and preserve the file
-comments. [See notes below about LIMITATIONS]
+The *filecopy* commands will copy/move the file to a destination, and 
+preserve the file comments. [See notes below about LIMITATIONS]
 
 The *cleanup* commands can clean up the history of comments in the database.
 
 # LIMITATIONS
 
-The file comments are located in two locations: a database, and in the xattr properties
-of the file. Each of these storage locations has its own benefits and limitations:
+The file comments are located in two locations: a database, and in the xattr 
+properties of the file. Each of these storage locations has its own benefits 
+and limitations:
 
 ## xattr
 
-Comments stored in the xattr properties can be copied/moved with the file, if you
+Because _xattr_ comments are bound to the filesystem, other command line tools 
+may be used to manage them. Comments stored in the _xattr_ properties can be 
+copied/moved with the file, if you
 use the correct options for _cp_. The _mv_ utility automatically preserves _xattr_.
 Other programs can also be coerced into perserving _xattr_ properties:
 
@@ -73,53 +81,64 @@ Other programs can also be coerced into perserving _xattr_ properties:
 * tar
 * mksquashfs
 
-Not all file systems support xattr properties (vfat/exfat does not). 
+Not all file systems support _xattr_ properties (vfat/exfat does not). 
 
-The current implementation of _sshfs_ and _scp_ do not support the copy of _xattr_ properties. 
-If you want to copy files to a remote machine and include the _xattr_ comments, use _rsync_ with the _-X_ option. Or _tar_ of course.
+The current implementation of _sshfs_ and _scp_ do not support the copy of 
+_xattr_ properties. If you want to copy files to a remote machine and 
+include the _xattr_ comments, use _rsync_ with the _-X_ option. Or _tar_ of course.
 
-Some editing apps (like _vim_) will create a new file when saving the data, which orphans the _xattr_ comments. For these apps, use the _database_ system.
+Some editing apps (like _vim_) will create a new file when saving the data, 
+which orphans the _xattr_ comments. For these apps, use the _database_ system.
+
+Of course, once you start to manipulate _xattr_ comments outside of the **dirnotes**
+programs, the _xattr_ comments will become out of sync with the _database_ comments.
 
 
 ## database
 
-Comments stored in the database work for all filesystem types (including vfat/exfat/sshfs)
-Comments are personalized to the _current user_. 
-Another user on the same system will not see these comments.
+Comments stored in the database work for all filesystem types 
+(including vfat/exfat/sshfs)
+Comments are usually stored in a _per user_ database; thus another user on 
+the same system will not see these comments.
 
 Files are indexed by their complete path name. Removable filesystems should be
 mounted in a consistent way, so that the complete path name is reproducable.
 
+If you are using _sshfs_, you can use the **dirnotes** programs to copy a file, 
+and the database comments will work properly.
+
 Comments stored in the database do not travel with the files when
- they are moved or copied, unless using the dirnotes-family of tools.
+they are moved or copied outside of using the **dirnotes**-family of tools.
 
-If you are using _sshfs_, you can use the 'dirnotes' programs to copy a file, and the database comments will work properly.
 
 # OPTIONS
   
+**-a**
+: append to a comment on a file
+
 **-c**
 : add a comment to a file
 
 **-C**
-: attempt to copy the file(s) and comments to a destination; if multiple files are copied, the destination must be a directory
+: attempt to copy the file(s) and comments to a destination; if multiple files are copied, the destination must be a directory; the last argument is the destination
 
 **-d**
 : use database comments as the primary source; cannot be used with **-x**
 
+**-D**
+: print debugging information
+
 **-e**
-: erase the comments on a file
+: erase the comments on the file(s)
 
 **-h** **--help**
 : display help and usage
 
 **-H**
-: output the history of comments for this file(s)
-
-**-i**  
-: add a comment to a file; if the comment is not in the command, accept it from stdin
+: output the history of comments for the file(s)
 
 **-j**
-: output (to stdio) the file comment in .json format 
+: output (to stdout) the file comment in .json format 
 
 **-l**
 : list all files, including those without _dirnotes_ comments
@@ -131,7 +150,7 @@ If you are using _sshfs_, you can use the 'dirnotes' programs to copy a file, an
 : output only the comment; this may be useful for scripting
 
 **-v**
-: also include the comment author and comment date in the output
+: list full path names, also include the comment author and date in the output
 
 **-V** **--version**
 : display version number
@@ -139,8 +158,11 @@ If you are using _sshfs_, you can use the 'dirnotes' programs to copy a file, an
 **-x**
 : use xattr comments as the primary source; connot be used with **-d**
 
+**-y**
+: allow file overwrite for **-C** copy or **-M** move
+
 **-z**
-: erase history comments associated with this file; keep the current comment; if no file is specified, erase the history comments for all files in the current directory
+: erase history comments associated with the file(s); keep the current comment
 
 **-Z**
 : erase all the historic comments in the user's database
@@ -149,19 +171,24 @@ If you are using _sshfs_, you can use the 'dirnotes' programs to copy a file, an
 
 To display the comment for a file:
 
-> $ dirnotes-cli filename.txt  
-> filename.txt: notes on the car repair
+> <code>$ dirnotes-cli car_notes.txt</code>  
+> <code>car_notes.txt: 'notes on the car repair'</code>
 
 To extract _only_ the comment from a file, use the _-n_ option (useful for scripts):
 
-> $ dirnotes-cli -n filename.txt  
-> notes on the car repair
+> <code>$ dirnotes-cli -n car_notes.txt</code>    
+> <code>notes on the car repair</code>
 
 Or, in json format:
 
-> $ dirnotes-cli -j filename.txt  
-> [{"file": "filename.txt", "comment": "notes on the car repair"}]
+> <code>$ dirnotes-cli -j car_notes.txt</code>  
+> <code>[{"file": "car_notes.txt", "comment": "notes on the car repair"}]</code>
+
+To append to a comment:
 
+> <code>$ dirnotes-cli -a 'first quote: \$1,400' car_notes.txt</code>  
+> <code>$ dirnotes-cli car_notes.txt</code>  
+> <code>car_notes.txt: 'notes on the car repair; first quote: $1,400'</code>
 
 
 # SEE ALSO
@@ -179,9 +206,10 @@ The **dirnotes-cli** program provides command line access, and can be scripted.
 By default, the file **~/.dirnotes.conf** will be used to load the user's config. There are
 three attributes described in that file that are important:
 
-> * xattr_tag  
-> * database  
-> * start_mode
+> * xattr_tag  [default: user.xdg.comment]   
+> * database   [default: ~/.local/share/dirnotes/dirnotes.db]  
+> * start_mode [default: xattr]  
 
+The default location for the config file is ~/.config/dirnotes/dirnotes.conf
 
 

dirnotes-tui

@@ -1,10 +1,11 @@
 #!/usr/bin/env python3 
 
-# TOTO: when changing directories, keep the sort order?
-# TODO: fix the 'reload' function....need 'this_dir' in Files class
 # TODO: write color scheme
 # TODO: re-read date/author to xattr after an edit
 # TODO: consider adding h,j,k,l movement
+# TODO: change move command to 'v', change mode to 'm', drop copy-comments
+# TODO: bug: enter db mode, type E to edit a comment, we get the xattr version!!
+# TODO: try to clear a comment, left with ' '
 
 # scroll
 # up/down - change focus, at limit: move 1 line,
@@ -23,16 +24,13 @@ import curses, sqlite3, curses.textpad
 import logging, getpass, argparse
 import json
 
-VERSION = "1.7"
+VERSION = "1.9"
 # these may be different on MacOS
 xattr_comment = "user.xdg.comment"
 xattr_author  = "user.xdg.comment.author"
 xattr_date    = "user.xdg.comment.date"
 DATE_FORMAT   = "%Y-%m-%d %H:%M:%S"
 
-# convert the ~/ form to a fully qualified path
-# database_name = "~/.dirnotes.db"
-
 mode_names = {"db":"<Database mode> ","xattr":"<Xattr mode>"}
 modes = ("db","xattr")
 mode = "db"
@@ -62,18 +60,6 @@ CMD_CD     = ord('\n')
 # ~/.dirnotes.db or /var/lib/dirnotes.db
 # at usage time, check for ~/.dirnotes.db first
 
-DEFAULT_CONFIG_FILE = "~/.dirnotes.conf"
-
-# config
-#    we could store the config in the database, in a second table
-#    or in a .json file
-DEFAULT_CONFIG = {"xattr_tag":"user.xdg.comment",
-  "database":"~/.dirnotes.db",
-  "start_mode":"xattr",
-  "options for database":("~/.dirnotes.db","/etc/dirnotes.db"),
-  "options for start_mode":("db","xattr")
-}
-
 ### colors
 
 CP_TITLE  = 1
@@ -92,38 +78,10 @@ COLOR_THEME = ''' { "heading": ("yellow","blue"),
 now = time.time()
 YEAR = 3600*24*365
 
-#
-# deal with the odd aspects of MacOS
-#
-#   xattr is loaded as a separate library, with different argument names
-#   to set/get the comments in Finder, we need to call osascript
-#
-
-if sys.platform == "darwin":
-  # we get xattr from PyPi
-  try:
-    import xattr
-  except:
-    print("This program requires the 'xattr' package. Please use 'pip3 install xattr' to install the package")
-    sys.exit(1)
-  def do_setxattr(file_name, attr_name, attr_content):
-    if attr_name.endswith("omment"):
-      p = Popen(['osascript','-'] + f'tell application "Finder" to set comment of (POSIX file "{file_name}" as alias) to "{attr_content}"')
-      p.communicate()
-    else:
-      xattr.setxattr(file_name, attr_name, plistlib.dumps(attr_content))
-
-  os.setxattr = do_setxattr
-  #os.setxattr = xattr.setxattr
-  def do_getxattr(file_name, attr_name):
-    attr_data = xattr.getxattr(file_name, attr_name)
-    if attr_name.endswith("omment"):
-      return plistlib.loads(attr_data)
-    else:
-      return attr_data
-  os.getxattr = xattr.getxattr
-else:
-  pass
+verbose = None
+def print_v(*a):
+  if verbose:
+    print(*a)
 
 class Pane:
   ''' holds the whole display: handles file list directly,
@@ -142,11 +100,12 @@ class Pane:
 
       most methods take y=0..h-1 where y is the line number WITHIN the borders
   '''
-  def __init__(self, win, curdir, files):
+  def __init__(self, win, curdir, files, start_file = None):
     self.curdir = curdir
     self.cursor = None
     self.first_visible = 0
     self.nFiles = len(files)
+    self.start_file = start_file
     
     self.h, self.w = win.getmaxyx()
     
@@ -184,18 +143,20 @@ class Pane:
  
     if self.some_comments_differ:
       self.setStatus("The xattr and database comments differ where shown in green")
-    #  time.sleep(2)
+    else:
+      self.setStatus("")
 
     self.file_pad.refresh(self.first_visible,0, 2,1, self.h-3,self.w-2)
  
   def refill(self):
     self.win.bkgdset(' ',curses.color_pair(CP_BODY))
     self.win.erase()
-    self.win.border()  # TODO: or .box() ?
+    self.win.box() 
     h,w = self.win.getmaxyx()
     self.win.addnstr(0,3,os.path.realpath(self.curdir),w-4)
-    n = len(files.getMasterComment())
-    self.win.addnstr(0,w-n-1,files.getMasterComment(),w-n-1)  # TODO: fix
+    mc = files.getMasterComment()
+    if mc:
+      self.win.addnstr(0,w-len(mc)-1,files.getMasterComment(),w-len(mc)-1)
     self.win.attron(COLOR_TITLE | curses.A_BOLD)
     self.win.addstr(1,1,'Name'.center(self.sep1-1))
     self.win.addstr(1,self.sep1+2,'Size')
@@ -214,27 +175,31 @@ class Pane:
     # and display the file_pan
     if self.cursor == None:
       self.cursor = 0
+      if self.start_file:  # if the command line had a file, find it and highlight it....once
+        for i,f in enumerate(files):
+          if f.getDisplayName() == self.start_file:
+            self.cursor = i
+      self.start_file = None
     self.focus_line()
 
   def fill_line(self,y):
     #logging.info(f"about to add {self.w-2} spaces at {y} to the file_pad size: {self.file_pad.getmaxyx()}")
-    # TODO: why do we have to have one extra line in the pad?
     f = files[y]  
     self.file_pad.addstr(y,0,' ' * (self.w-2))
-    self.file_pad.addnstr(y,0,f.getFileName(),self.sep1-1)
-    self.file_pad.addstr(y,self.sep1,makeSize(f.size))
-    self.file_pad.addstr(y,self.sep2,makeDate(f.date))
-
-    dbComment = f.getDbComment()
-    xattrComment = f.getXattrComment()
-    comment = xattrComment if mode=="xattr" else dbComment
-    if dbComment != xattrComment:
+    self.file_pad.addnstr(y,0,f.getDisplayName(),self.sep1-1)
+    self.file_pad.addstr(y,self.sep1,UiHelper.getShortSize(f))
+    self.file_pad.addstr(y,self.sep2,UiHelper.getShortDate(f.date))
+
+    comment = f.getComment(mode) or ''
+    other   = f.getOtherComment(mode) or ''
+    logging.info(f"file_line, comments are <{comment}> and <{other}>  differ_flag:{self.some_comments_differ}")
+    if comment == other:
+      self.file_pad.addnstr(y,self.sep3,comment,self.w-self.sep3-2)
+    else:
       self.some_comments_differ = True
       self.file_pad.attron(COLOR_HELP)
-      self.file_pad.addnstr(y,self.sep3,comment,self.w-self.sep3-2)
+      self.file_pad.addnstr(y,self.sep3,comment or '       ',self.w-self.sep3-2)
       self.file_pad.attroff(COLOR_HELP)
-    else:
-      self.file_pad.addnstr(y,self.sep3,comment,self.w-self.sep3-2)
 
     self.file_pad.vline(y,self.sep1-1,curses.ACS_VLINE,1)
     self.file_pad.vline(y,self.sep2-1,curses.ACS_VLINE,1)
@@ -279,90 +244,63 @@ class Pane:
     self.statusbar.addnstr(" " + data,w-x-1)
     self.statusbar.refresh()
 
-# two helpers to format the date & size visuals
-def makeDate(when):
-  ''' arg when is epoch seconds in localtime '''
-  diff = now - when
-  if diff > YEAR:
-    fmt = "%b %e  %Y"
-  else:
-    fmt = "%b %d %H:%M"
-  return time.strftime(fmt, time.localtime(when))
-
-def makeSize(size):
-  if size == FileObj.FILE_IS_DIR:
-    return " <DIR> "
-  elif size == FileObj.FILE_IS_LINK:
-    return " <LINK>"
-  log = int((math.log10(size+1)-2)/3)
-  s = " KMGTE"[log]
-  base = int(size/math.pow(10,log*3))
-  return f"{base}{s}".strip().rjust(7)
-  
-
 ## to hold the FileObj collection
 
 class Files():
-  def __init__(self,directory):
-    self.directory = FileObj(directory)
+  def __init__(self,directory,db):
+    self.db = db
 
-    self.files = []
-    if directory != '/':
-      self.files.append(FileObj(directory + "/.."))
-    # TODO: switch to os.scandir()
-    f = os.listdir(directory)
-    for i in f:
-      self.files.append(FileObj(directory + '/' + i))
-    self.sort()
+    if not os.path.isdir(directory):
+      errorBox(f"the command line argument: {directory} is not a directory; starting in the current directory")
+      directory = '.'
+    self.directory = FileObj(directory,self.db)
 
-    self.db = None
     try:
-      self.db = sqlite3.connect(database_name)
-      c = self.db.cursor()
-      c.execute("select * from dirnotes")
-    except sqlite3.OperationalError:
-      # TODO: problem with database....create one?
-      c.execute("create table dirnotes (name TEXT, date DATETIME, size INTEGER, comment TEXT, comment_date DATETIME, author TEXT)")
+      current, dirs, non_dirs = next(os.walk(directory))
+    except:
+      errorBox(f"{directory} is not a valid directory")
+      raise
+    if current != '/':
+      dirs.insert(0,"..")
 
-    try:
-      c = self.db.cursor()
-      self.directory.loadDbComment(c)
-      for f in self.files:
-        f.loadDbComment(c)
-    except sqlite3.OperationalError:
-      errorBox("serial problem with the database")
+    self.files = []
+    for f in dirs + non_dirs:
+      self.files.append(FileObj(os.path.join(current,f),self.db))
+    self.sort()
 
 
   def sortName(a):
     ''' when sorting by name put the .. and other <dir> entries first '''
-    if a.getFileName() == '..':
+    if a.getDisplayName() == '../':
       return "\x00"
     if a.isDir():
-      return ' ' + a.getFileName()
+      return ' ' + a.getDisplayName()
     # else:
-    return a.getFileName()
+    return a.getDisplayName()
 
   def sortDate(a):
-    if a.getFileName() == '..':
+    if a.getDisplayName() == '../':
       return 0
     return a.getDate()
 
   def sortSize(a):
-    if a.getFileName() == '..':
-      return 0
+    if a.getDisplayName() == '../':
+      return -2
+    if a.isDir() or a.isLink() or a.isSock():
+      return -1
     return a.getSize()
 
   def sortComment(a):
-    return a.getComment()
+    return a.getComment(mode) or '~'
 
-  sort_mode = sortName
+  sortFunc = sortName
   def sort(self):
-    self.files.sort(key = Files.sort_mode)
+    self.files.sort(key = Files.sortFunc)
 
   def getCurDir(self):
     return self.directory
   def getMasterComment(self):
-    return self.directory.getComment()
+    return self.directory.getComment(mode)
 
   ## accessors ##
   def __len__(self):
@@ -373,133 +311,261 @@ class Files():
     return self.files.__iter__()
       
 def errorBox(string):
-  werr = curses.newwin(3,len(string)+8,5,5)
-  werr.bkgd(' ',COLOR_ERROR)
-  werr.clear()
-  werr.border()
-  werr.addstr(1,1,string)
-  werr.getch()  # any key
-  del werr
+  if curses_running:
+    werr = curses.newwin(3,len(string)+8,5,5)
+    werr.bkgd(' ',COLOR_ERROR)
+    werr.clear()
+    werr.box()
+    werr.addstr(1,1,string)
+    werr.timeout(3000)
+    werr.getch()  # any key
+    del werr
+  else:
+    print(string)
+    time.sleep(3)
   
+############# the dnDataBase, UiHelper and FileObj code is shared with other dirnotes programs
 
-## one for each file
-## a special one called .. exists for the parent
-class FileObj():
-  FILE_IS_DIR = -1
-  FILE_IS_LINK = -2
-  def __init__(self, fileName):
-    self.fileName = os.path.realpath(fileName)
-    self.displayName = '..' if fileName.endswith('/..') else os.path.split(fileName)[1] 
-    s = os.lstat(fileName)
-    self.date = s.st_mtime
-    if stat.S_ISDIR(s.st_mode):
-      self.size = FileObj.FILE_IS_DIR
-    elif stat.S_ISLNK(s.st_mode):
-      self.size = FileObj.FILE_IS_LINK
-    else:
-      self.size = s.st_size
-    self.xattrComment = ''
-    self.xattrAuthor = None
-    self.xattrDate = None
-    self.dbComment = ''
-    self.dbAuthor = None
-    self.dbDate = None
-    self.commentsDiffer = False
+DEFAULT_CONFIG_FILE = "~/.dirnotes.conf"
+
+# config
+#    we could store the config in the database, in a second table
+#    or in a .json file
+DEFAULT_CONFIG = {"xattr_tag":"user.xdg.comment",
+  "database":"~/.dirnotes.db",
+  "start_mode":"xattr",
+  "options for database":("~/.dirnotes.db","/etc/dirnotes.db"),
+  "options for start_mode":("db","xattr")
+}
+
+class ConfigLoader:    # singleton
+  def __init__(self, configFile):
+    configFile = os.path.expanduser(configFile)
+    try:
+      with open(configFile,"r") as f:
+        config = json.load(f)
+    except json.JSONDecodeError:
+      errorBox(f"problem reading config file {configFile}; check the JSON syntax")
+      config = DEFAULT_CONFIG
+    except FileNotFoundError:
+      errorBox(f"config file {configFile} not found; using the default settings")
+      config = DEFAULT_CONFIG
+      try:
+        with open(configFile,"w") as f:
+          json.dump(config,f,indent=4)
+      except:
+        errorBox(f"problem creating the config file {configFile}")
+    self.dbName = os.path.expanduser(config["database"])
+    self.mode = config["start_mode"]    # can get over-ruled by the command line options
+    self.xattr_comment = config["xattr_tag"]
+
+class DnDataBase:
+  ''' the database is flat
+    fileName: fully qualified name
+    st_mtime: a float
+    size: a long
+    comment: a string
+    comment_time: a float, the time of the comment save
+    author: the username that created the comment
+
+    this object: 1) finds or creates the database
+      2) determine if it's readonly
+
+    TODO: the database is usually associated with a user, in $XDG_DATA_HOME (~/.local/share/) 
+    TODO: if the database is not found, create it in $XDG_DATA_DIRS (/usr/local/share/)
+      make it 0666 permissions (rw-rw-rw-)
+  '''
+  def __init__(self,dbFile):
+    '''try to open the database; if not found, create it'''
+    try:
+      self.db = sqlite3.connect(dbFile)
+    except sqlite3.OperationalError:
+      logging.error(f"Database {dbFile} not found")
+      raise
+ 
+    # create new database if it doesn't exist
+    try:
+      self.db.execute("select * from dirnotes")
+    except sqlite3.OperationalError:
+      print_v(f"Table dirnotes created")
+      self.db.execute("create table dirnotes (name TEXT, date DATETIME, size INTEGER, comment TEXT, comment_date DATETIME, author TEXT)")
+      self.db.execute("create index dirnotes_i on dirnotes(name)") 
+      # at this point, if a shared database is required, somebody needs to set perms to 0o666
+  
+    self.writable = True
     try:
-      self.xattrComment = os.getxattr(fileName, xattr_comment, follow_symlinks=False).decode()
-      self.xattrAuthor  = os.getxattr(fileName, xattr_author, follow_symlinks=False).decode()
-      self.xattrDate    = os.getxattr(fileName, xattr_date, follow_symlinks=False).decode()
-      self.commentsDiffer = True if self.xattrComment == self.dbComment else False
-    except:  # no xattr comment
-      pass
+      self.db.execute("pragma user_verson=0")
+    except sqlite3.OperationalError:
+      self.writable = False
+
+class UiHelper:
+  @staticmethod
+  def epochToDb(epoch):
+    return time.strftime(DATE_FORMAT,time.localtime(epoch))
+  @staticmethod
+  def DbToEpoch(dbTime):
+    return time.mktime(time.strptime(dbTime,DATE_FORMAT))
+  @staticmethod
+  def getShortDate(longDate):
+    now = time.time()
+    diff = now - longDate
+    if diff > YEAR:
+      fmt = "%b %e  %Y"
+    else:
+      fmt = "%b %e %H:%M"
+    return time.strftime(fmt, time.localtime(longDate))
+  @staticmethod
+  def getShortSize(fo):
+    if fo.isDir():
+      return " <DIR> "
+    elif fo.isLink():
+      return " <LINK>"
+    size = fo.getSize()
+    log = int((math.log10(size+1)-2)/3)
+    s = " KMGTE"[log]
+    base = int(size/math.pow(10,log*3))
+    return f"{base}{s}".strip().rjust(7)
+
+
+## one for each file
+## and a special one for ".." parent directory
+class FileObj:
+  """  The FileObj knows about both kinds of comments. """
+  def __init__(self, fileName, db):
+    self.fileName = os.path.abspath(fileName)     # full path; dirs end WITHOUT a terminal /
+    self.stat = os.lstat(self.fileName)
+    self.displayName = os.path.split(fileName)[1] # base name; dirs end with a /
+    if self.isDir():
+      if not self.displayName.endswith('/'):
+        self.displayName += '/'
+    self.date = self.stat.st_mtime
+    self.size = self.stat.st_size 
+    self.db = db
 
   def getName(self):
+    """ returns the absolute pathname """
     return self.fileName
-  def getFileName(self):
+  def getDisplayName(self):
+    """ returns just this basename of the file; dirs end in / """
     return self.displayName
 
-  # with an already open database cursor
-  def loadDbComment(self,c):
-    c.execute("select comment,author,comment_date from dirnotes where name=? and comment<>'' order by comment_date desc",(self.fileName,))
-    a = c.fetchone()
-    if a:
-      self.dbComment, self.dbAuthor, self.dbDate = a
-      self.commentsDiffer = True if self.xattrComment == self.dbComment else False
-  
+  def getDbData(self):
+    """ returns (comment, author, comment_date) """
+    if not hasattr(self,'dbCommentAuthorDate'):
+      cad = self.db.execute("select comment, author, comment_date from dirnotes where name=? order by comment_date desc",(self.fileName,)).fetchone()
+      self.dbCommentAuthorDate = cad if cad else (None, None, None)
+    return self.dbCommentAuthorDate
   def getDbComment(self):
-    return self.dbComment
-  def getDbAuthor(self):
-    return self.dbAuthor
-  def getDbDate(self):
-    return self.dbDate
+    return self.getDbData()[0]
+
+  def getXattrData(self):
+    """ returns (comment, author, comment_date) """
+    if not hasattr(self,'xattrCommentAuthorDate'):
+      c = a = d = None
+      try:
+        c = os.getxattr(self.fileName, xattr_comment, follow_symlinks=False).decode()
+        a = os.getxattr(self.fileName, xattr_author, follow_symlinks=False).decode()
+        d = os.getxattr(self.fileName, xattr_date, follow_symlinks=False).decode()
+      except:  # no xattr comment
+        pass
+      self.xattrCommentAuthorDate = c,a,d
+    return self.xattrCommentAuthorDate
+  def getXattrComment(self):
+    return self.getXattrData()[0]
+
   def setDbComment(self,newComment):
-    try:
-      self.db = sqlite3.connect(database_name)
-    except sqlite3.OperationalError:
-      logging.info(f"database {database_name} not found")
-      raise OperationalError
-    c = self.db.cursor()
+    # how are we going to hook this?
+    #if not self.db.writable:
+    #  errorBox("The database is readonly; you cannot add or edit comments")
+    #  return
     s = os.lstat(self.fileName)
     try:
-      c.execute("insert into dirnotes (name,date,size,comment,comment_date,author) values (?,datetime(?,'unixepoch','localtime'),?,?,datetime(?,'unixepoch','localtime'),?)",
-          (os.path.abspath(self.fileName), s.st_mtime, s.st_size, 
+      print_v(f"setDbComment db {self.db}, file: {self.fileName}")
+      self.db.execute("insert into dirnotes (name,date,size,comment,comment_date,author) values (?,datetime(?,'unixepoch','localtime'),?,?,datetime(?,'unixepoch','localtime'),?)",
+          (self.fileName, s.st_mtime, s.st_size,
           str(newComment), time.time(), getpass.getuser()))
       self.db.commit()
-      logging.info(f"database write for {self.fileName}")
-      self.dbComment = newComment
+      self.dbCommentAuthorDate = newComment, getpass.getuser(), UiHelper.epochToDb(time.time())
     except sqlite3.OperationalError:
-      logging.info("database is locked or unwritable")
+      print_v("database is locked or unwritable")
       errorBox("the database that stores comments is locked or unwritable")
-      self.commentsDiffer = True if self.xattrComment == self.dbComment else False
-    
-  def getXattrComment(self):
-    return self.xattrComment
-  def getXattrAuthor(self):
-    return self.xattrAuthor
-  def getXattrDate(self):
-    logging.info(f"someone accessed date on {self.fileName} {self.xattrDate}")
-    return self.xattrDate
+
   def setXattrComment(self,newComment):
-    logging.info(f"set comment {newComment} on file {self.fileName}")
+    print_v(f"set comment {newComment} on file {self.fileName}")
     try:
       os.setxattr(self.fileName,xattr_comment,bytes(newComment,'utf8'),follow_symlinks=False)
       os.setxattr(self.fileName,xattr_author,bytes(getpass.getuser(),'utf8'),follow_symlinks=False)
       os.setxattr(self.fileName,xattr_date,bytes(time.strftime(DATE_FORMAT),'utf8'),follow_symlinks=False)
-      self.xattrAuthor = getpass.getuser()
-      self.xattrDate = time.strftime(DATE_FORMAT)      # alternatively, re-instantiate this FileObj
-      self.xattrComment = newComment
-      self.commentsDiffer = True if self.xattrComment == self.dbComment else False
+      self.xattrCommentAuthorDate = newComment, getpass.getuser(), time.strftime(DATE_FORMAT) 
       return True
     # we need to move these cases out to a handler 
     except Exception as e:
-      errorBox("problem setting the comment on file %s" % self.getName())
-      errorBox("error "+repr(e))
-      ## todo: elif file.is_sym() the kernel won't allow comments on symlinks....stored in database
-      if self.size == FileObj.FILE_IS_LINK:
-        errorBox("Linux does not allow comments on symlinks; comment is stored in database")
+      if self.isLink():
+        errorBox("Linux does not allow xattr comments on symlinks; comment is stored in database")
+      elif self.isSock():
+        errorBox("Linux does not allow comments on sockets; comment is stored in database")
       elif os.access(self.fileName, os.W_OK)!=True:
         errorBox("you don't appear to have write permissions on this file")
         # change the listbox background to yellow
-        self.displayBox.notifyUnchanged()               
       elif "Errno 95" in str(e):
         errorBox("is this a VFAT or EXFAT volume? these don't allow comments")
       return False
 
-  def getComment(self):
-    return self.getDbComment() if mode == "db" else self.getXattrComment()
-  def getOtherComment(self):
+  def getComment(self,mode):
+    return self.getDbComment() if mode == "db"    else self.getXattrComment()
+  def getOtherComment(self,mode):
     return self.getDbComment() if mode == "xattr" else self.getXattrComment()
+  def getData(self,mode):
+    """ returns (comment, author, comment_date) """
+    return self.getDbData()    if mode == "db"    else self.getXattrData()
+  def getOtherData(self,mode):
+    """ returns (comment, author, comment_date) """
+    return self.getDbData()    if mode == "xattr" else self.getXattrData()
+
   def getDate(self):
     return self.date
   def getSize(self):
     return self.size
   def isDir(self):
-    return self.size == self.FILE_IS_DIR
-
-##########  dest folder picker ###############
+    return stat.S_ISDIR(self.stat.st_mode)
+  def isLink(self):
+    return stat.S_ISLNK(self.stat.st_mode)
+  def isSock(self):
+    return stat.S_ISSOCK(self.stat.st_mode)
+
+  def copyFile(self, destDir):
+    # NOTE: this method copies the xattr (comment + old author + old date) 
+    #       but creates new db (comment + this author + new date)
+    if stat.S_ISREG(self.stat.st_mode):
+      dest = os.path.join(destDir,self.displayName)
+      try:
+        print_v("try copy from",self.fileName,"to",dest)
+        shutil.copy2(self.fileName, dest)
+      except:
+        errorBox(f"file copy to <{dest}> failed; check permissions")
+        return
+      f = FileObj(dest, self.db)
+      f.setDbComment(self.getDbComment())
+  def moveFile(self, destDir):
+    # NOTE: this method moves the xattr (comment + old author + old date)
+    #       but creates new db (comment + this author + new date)
+    src  = self.fileName
+    dest = os.path.join(destDir, self.displayName)
+    # move preserves dates & chmod/chown & xattr
+    print_v(f"move from {self.fileName} to {destDir}")
+    try:
+      shutil.move(src, dest)
+    except:
+      ErrorBox(f"file move to <{dest}> failed; check permissions")
+      return
+    # and copy the database record
+    f = FileObj(dest,files.db)
+    f.setDbComment(self.getDbComment())  
+    
+##########  dest directory picker ###############
 # returns None if the user hits <esc>
 #     the dir_pad contents are indexed from 0,0, matching self.fs
-class showFolderPicker:
+class showDirectoryPicker:
   def __init__(self,starting_dir,title):
     self.selected = None
     self.title = title
@@ -511,7 +577,7 @@ class showFolderPicker:
     self.h, self.w = self.W.getmaxyx()
     self.W.keypad(True)
     #self.W.clear()
-    self.W.border()
+    self.W.box()
     self.W.addnstr(0,1,self.title,self.w-2)
     self.W.addstr(self.h-1,1,"<Enter> to select or change dir, <esc> to exit")
     self.W.refresh()
@@ -593,7 +659,7 @@ def paint_dialog(b_color,data):
   w = curses.newwin(len(lines)+2,n+3,5,5)
   w.bkgd(' ',b_color)
   w.clear()
-  w.border()
+  w.box()
   for i,d in enumerate(lines):
     w.addnstr(i+1,1,d,n)
   #w.refresh I don't know why this isn't needed :(
@@ -673,8 +739,6 @@ def show_help2():
   c = w.getch()
   del w
 
-#TODO: fix this to paint_dialog
-#TODO: fix to allow upper/lower case responses
 sort_string = """
 Select sort order: 
  
@@ -701,19 +765,11 @@ Comments detail:
 def show_detail(f):
   global mode
   h = paint_dialog(COLOR_HELP,detail_string)
-  if mode=="xattr":
-    h.addstr(1,20,"from xattrs")
-    c = f.getXattrComment()
-    a = f.getXattrAuthor()
-    d = time.ctime(f.getXattrDate())
-  else:
-    h.addstr(1,20,"from database")
-    c = f.getDbComment()
-    a = f.getDbAuthor()
-    d = f.getDbDate()
-  h.addnstr(2,12,c,h.getmaxyx()[1]-13)
-  h.addstr(3,12,a if a else "<not set>")
-  h.addstr(4,12,d if d else "<not set>")
+  c,a,d = f.getData(mode)   # get all three, depending on the current mode
+  h.addstr(1,20,"from xattrs" if mode=="xattr" else "from database")
+  h.addnstr(2,12,c or "<not set>",h.getmaxyx()[1]-13)
+  h.addstr(3,12,a or "<not set>")
+  h.addstr(4,12,d or "<not set>")
   h.refresh()
   c = h.getch()
   del h
@@ -731,7 +787,7 @@ def edit_fn(c):
     return 7
   return c
 
-def main(w, cwd):
+def main(w, cwd, database_file, start_file):
   global files, edit_done, mode
   global COLOR_TITLE, COLOR_BODY, COLOR_FOCUS, COLOR_ERROR, COLOR_HELP
 
@@ -750,10 +806,12 @@ def main(w, cwd):
   COLOR_DIFFER = curses.color_pair(CP_DIFFER)
   logging.info(f"COLOR_DIFFER is {COLOR_DIFFER}")
 
-  files = Files(cwd)
+  db = DnDataBase(database_file).db
+
+  files = Files(cwd,db)
   logging.info(f"got files, len={len(files)}")
 
-  mywin = Pane(w,cwd,files)
+  mywin = Pane(w,cwd,files,start_file = start_file)
     
   showing_edit = False
 
@@ -770,13 +828,13 @@ def main(w, cwd):
     elif c == CMD_SORT:
       c = show_sort()
       if c == ord('s') or c == ord('S'):
-        Files.sort_mode = Files.sortSize
+        Files.sortFunc = Files.sortSize
       elif c == ord('n') or c == ord('N'):
-        Files.sort_mode = Files.sortName
+        Files.sortFunc = Files.sortName
       elif c == ord('d') or c == ord('D'):
-        Files.sort_mode = Files.sortDate
+        Files.sortFunc = Files.sortDate
       elif c == ord('c') or c == ord('C'):
-        Files.sort_mode = Files.sortComment
+        Files.sortFunc = Files.sortComment
       files.sort()
       mywin.refill()
       mywin.refresh()
@@ -805,30 +863,30 @@ def main(w, cwd):
 
     elif c == CMD_RELOAD:
       where = files.getCurDir().fileName
-      files = Files(where)
+      files = Files(where,db)
       mywin = Pane(w,where,files)
 
     elif c == CMD_CD:
       f = files[mywin.cursor]
       if f.isDir():
         cwd = f.getName()
-        logging.info(f"CD change to {cwd}")
-        files = Files(cwd)
+        print_v(f"CD change to {cwd}")
+        files = Files(cwd,db)
         mywin = Pane(w,cwd,files)
         # TODO: should this simply re-fill() the existing Pane instead of destroy?
 
     elif c == CMD_EDIT:
       showing_edit = True
       edit_window = curses.newwin(5,(4*mywin.w) // 5,mywin.h // 2 - 3, mywin.w // 10)
-      edit_window.border()
+      edit_window.box()
       edit_window.addstr(0,1,"<enter> when done, <esc> to cancel")
       he,we = edit_window.getmaxyx()
       edit_sub = edit_window.derwin(3,we-2,1,1)
       
       f = files[mywin.cursor]
-      mywin.setStatus(f"Edit file: {f.getFileName()}")
+      mywin.setStatus(f"Edit file: {f.getName()}")
       existing_comment = f.getXattrComment()
-      edit_sub.addstr(0,0,existing_comment) 
+      edit_sub.addstr(0,0,existing_comment or '') 
       text = curses.textpad.Textbox(edit_sub)
       edit_window.refresh()
 
@@ -852,71 +910,55 @@ def main(w, cwd):
     elif c == CMD_CMNT_CP:
       # copy comments to the other mode
       cp_cmnt_ask = curses.newwin(6,40,5,5)
-      cp_cmnt_ask.border()
+      cp_cmnt_ask.box()
       cp_cmnt_ask.addstr(1,1,"Copy comments to ==> ")
       cp_cmnt_ask.addstr(1,22,"database" if mode=="xattr" else "xattr")
-      cp_cmnt_ask.addstr(2,1,"1  just this file")
-      cp_cmnt_ask.addstr(3,1,"a  all files with comments")
+      cp_cmnt_ask.addstr(2,1," 1  just this file")
+      cp_cmnt_ask.addstr(3,1," a  all files with comments")
       cp_cmnt_ask.addstr(4,1,"esc to cancel")
       cp_cmnt_ask.refresh()
        
       c = cp_cmnt_ask.getch()
-      # esc
-      if c!=ord('1') and c!=ord('a') and c!=ord('A'):
-        continue
-      # copy comments for one file or all
-      if c==ord('1'):
-        collection = [files[mywin.cursor]]
-      else:
-        collection = files
-      for f in collection:
-        if mode=="xattr":
-          if f.getXattrComment():
-            f.setDbComment(f.getXattrComment())
+      if c in (ord('1'), ord('a'), ord('A')):
+        # copy comments for one file or all
+        if c==ord('1'):
+          collection = [files[mywin.cursor]]
         else:
-          if f.getDbComment():
-            f.setXattrComment(f.getDbComment())
+          collection = files
+        for f in collection:
+          if mode=="xattr":
+            if f.getXattrComment():
+              f.setDbComment(f.getXattrComment())
+          else:
+            if f.getDbComment():
+              f.setXattrComment(f.getDbComment())
       mywin.refill()
       mywin.refresh()
 
     elif c == CMD_COPY:
-      if files[mywin.cursor].displayName == "..":
-        continue
-      if os.path.isdir(files[mywin.cursor].fileName):
-        errorBox(f"<{files[mywin.cursor].displayName}> is a directory. Copy not allowed")
+      if files[mywin.cursor].getDisplayName() == "../":
         continue
-      dest_dir = showFolderPicker(cwd,"Select folder for copy").value()
-      if dest_dir:
-        #errorBox(f"copy cmd to {dest_dir}")
-        src = cwd + '/' + files[mywin.cursor].displayName
-        dest = dest_dir + '/' + files[mywin.cursor].displayName
-        # copy2 preserves dates & chmod/chown & xattr
-        logging.info(f"copy from {src} to {dest_dir}")
-        shutil.copy2(src, dest_dir)
-        # and copy the database record
-        f = FileObj(dest)
-        f.setDbComment(files[mywin.cursor].getDbComment())
+      if files[mywin.cursor].isDir():
+        errorBox(f"<{files[mywin.cursor].getDisplayName()}> is a directory. Copy not allowed")
+      else:
+        dest_dir = showDirectoryPicker(cwd,"Select folder for copy").value()
+        if dest_dir:
+          files[mywin.cursor].copyFile(dest_dir)
       mywin.refresh() 
 
     elif c == CMD_MOVE:
-      if files[mywin.cursor].displayName == "..":
-        continue
-      if os.path.isdir(files[mywin.cursor].fileName):
-        errorBox(f"<{files[mywin.cursor].displayName}> is a directory. Move not allowed")
+      if files[mywin.cursor].getDisplayName() == "../":
         continue
-      dest_dir = showFolderPicker(cwd,"Select folder for move").value()
-      if dest_dir:
-        #errorBox(f"move cmd to {dest_dir}")      
-        src = cwd + '/' + files[mywin.cursor].displayName
-        dest = dest_dir + '/' + files[mywin.cursor].displayName
-        # move preserves dates & chmod/chown & xattr
-        logging.info(f"move from {src} to {dest_dir}")
-        shutil.move(src, dest_dir)
-        # and copy the database record
-        f = FileObj(dest)
-        f.setDbComment(files[mywin.cursor].getDbComment())  
-        files = Files(cwd)
-        mywin = Pane(w,cwd,files) # is this the way to refresh the main pane? TODO
+      if files[mywin.cursor].isDir():
+        errorBox(f"<{files[mywin.cursor].getDisplayName()}> is a directory. Move not allowed")
+      else:
+        dest_dir = showDirectoryPicker(cwd,"Select folder for move").value()
+        if dest_dir:
+          files[mywin.cursor].moveFile(dest_dir)
+          files = Files(cwd,db)
+          mywin = Pane(w,cwd,files) # is this the way to refresh the main pane? TODO
+      mywin.refresh()   # to clean up the errorBox or FolderPicker
+        
 
     elif c == curses.KEY_RESIZE:
       mywin.resize()
@@ -930,51 +972,36 @@ def pre_main():
   parser = argparse.ArgumentParser(description="Add comments to files")
   parser.add_argument('-c','--config',  dest='config_file', help="config file (json format)")
   parser.add_argument('-v','--version', action='version', version=f"dirnotes ver:{VERSION}")
-  parser.add_argument('directory', type=str, default='.', nargs='?',  help="directory to start")
+  parser.add_argument('-d','--db', action='store_true',help="start up in database mode")
+  parser.add_argument('-x','--xattr', action='store_true',help="start up in xattr mode")
+  parser.add_argument('directory', type=str, default='.', nargs='?',  help="directory or file to start")
   args = parser.parse_args()
   logging.info(args)
 
-  if args.config_file:
-    config_file = args.config_file
-  else:
-    config_file = DEFAULT_CONFIG_FILE
-  config_file = os.path.expanduser(config_file)
-
-  config = DEFAULT_CONFIG
-  try:
-    with open(config_file,"r") as f:
-      config = json.load(f)
-  except json.JSONDecodeError:
-    print(f"problem reading the config file {config_file}")
-    printf("please check the .json syntax")
-    time.sleep(2)
-  except FileNotFoundError:
-    print(f"config file {config_file} not found, using default settings & creating a default")
-    try:
-      with open(config_file,"w") as f:
-        json.dump(config,f,indent=4)
-    except:
-      print(f"problem creating file {config_file}")
-    time.sleep(2)
-
+  config = ConfigLoader(args.config_file or DEFAULT_CONFIG_FILE)
+  if args.db:
+    config.mode = "db"
+  if args.xattr:
+    config.mode = "xattr"
   # print(repr(config))
   # print("start_mode",config["start_mode"])
   
   return args, config
 
+curses_running = False
 args, config = pre_main()
 
-mode = config["start_mode"]
-xattr_comment = config["xattr_tag"]
-xattr_author  = config["xattr_tag"] + ".author"
-xattr_date    = config["xattr_tag"] + ".date"
-# change ~ tilde to username
-database_name = os.path.expanduser(config["database"])
-cwd = args.directory
+mode = config.mode
+xattr_comment = config.xattr_comment
+xattr_author  = config.xattr_comment + ".author"
+xattr_date    = config.xattr_comment + ".date"
+database_name = config.dbName
+if os.path.isdir(args.directory):
+  cwd, start_file = args.directory, None
+else:
+  cwd, start_file = os.path.split(args.directory)
 
-curses.wrapper(main, cwd)
+curses_running = True
+curses.wrapper(main, cwd or '.', database_name, start_file)
 
-# dirnotes database is name, date, size, comment, comment_date, author
 
-# symlinks: follow_symlinks should always be True, because symlinks in Linux
-#    can't have xattr....it appears to be the same in darwin