diff -Nru offlineimap-7.0.12+dfsg1/Changelog.md offlineimap-7.1.2+dfsg1/Changelog.md --- offlineimap-7.0.12+dfsg1/Changelog.md 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/Changelog.md 2017-07-10 14:58:50.000000000 +0000 @@ -15,6 +15,235 @@ * The following excerpt is only usefull when rendered in the website. {:toc} +### OfflineIMAP v7.1.2 (2017-07-10) + +#### Notes + +This release introduces better Davmail support, better reliability when in +IMAP/IMAP mode, better output on some errors, and minor fixes. The provided +systemd files are improved. + +The imaplib2 requirement is now v2.57. + +Remi Locherer is joining our tester team. Great! + +Starting with this release, the feedbacks from the testers are recorded in the +release notes, the git logs and the Changelog. Thanks to all of them for +improving the releases. + +This release was tested by: + +- benutzer193 +- Nicolas Sebrecht +- Remi Locherer + +#### Authors + +- Nicolas Sebrecht (20) +- Hugo Osvaldo Barrera (5) +- Alvaro Pereyra (1) +- benutzer193 (1) + +#### Features + +- contrib/release.py: consider positive feedbacks from testers. [Nicolas Sebrecht] +- Introduce the github CODEOWNERS file. [Nicolas Sebrecht] +- IMAP/IMAP: continue to sync if the local side does not return a valid UID on upload. [Nicolas Sebrecht] + +#### Fixes + +- folder/IMAP: introduce dedicated parsing for davmail (not supporting UIDPLUS). [Nicolas Sebrecht] +- offlineimap.conf: minor typo fix. [Alvaro Pereyra] +- Respect systemd conventions for timers. [Hugo Osvaldo Barrera] +- Use a pre-existing target for systemd services. [Hugo Osvaldo Barrera] +- Remove invalid systemd setting. [Hugo Osvaldo Barrera] +- systemd: remove unused watchdog functionality. [benutzer193] +- gitignore generated css file. [Nicolas Sebrecht] +- Changelog: fix syntax. [Nicolas Sebrecht] + +#### Changes + +- Increase imaplib2 requirement from v2.55 to v2.57. [Nicolas Sebrecht] +- folder/IMAP: improve the warning when we can't parse the returned UID. [Nicolas Sebrecht] +- Provide more details in error message when SSL fails on non-standard port. [Nicolas Sebrecht] +- Use basic logger (since systemd picks up stdout). [Hugo Osvaldo Barrera] +- Explain how to override systemd values. [Hugo Osvaldo Barrera] +- systemd: add documentation entry in configuration files. [Nicolas Sebrecht] +- offlineimap.conf: ssl must be disabled to force STARTTLS in some cases. [Nicolas Sebrecht] +- Advise singlethreadperfolder when offlineimap hangs. [Nicolas Sebrecht] +- offlineimap.conf: minor improvements. [Nicolas Sebrecht] +- contrib: more release automation. [Nicolas Sebrecht] +- MAINTAINERS: Remi Locherer joins the team of testers. [Nicolas Sebrecht] +- systemd: README: credit Hugo as contributor. [Nicolas Sebrecht] + + +### OfflineIMAP v7.1.1 (2017-05-28) + +#### Notes + +This release has some interesting fixes, including one for the Blinkenlights UI. + +Otherwise, there is no big change since the previous version. + +Furthermore, this release was tested by: + +- Remi Locherer + +#### Authors + +- Nicolas Sebrecht (17) +- Chris Coleman (1) +- Ilias Tsitsimpis (1) +- Maximilian Kaul (1) +- benutzer193 (1) +- Ævar Arnfjörð Bjarmason (1) + +#### Features + +- contrib: introduce a tool to produce the "upcoming notes". [Nicolas Sebrecht] +- contrib: secure HTTPS test internet is connected.. [Chris Coleman] +- Env info (used by -V and banner): add openssl version. [Nicolas Sebrecht] +- docs: learn to build html files for the manual pages. [Nicolas Sebrecht] + +#### Fixes + +- Acquire lock before updating the CursesLogHandler window. [Ilias Tsitsimpis] +- maxage: use the remote folder first to compute min_uid. [Nicolas Sebrecht] +- Fix systemd.timer: initialize timer after boot. [benutzer193] +- XOAUTH2: don't try this authentication method when not configured. [Nicolas Sebrecht] +- mbnames: don't duplicate entries in autorefresh mode. [Nicolas Sebrecht] +- docs: update the instructions for creating OAuth projects for GMail. [Ævar Arnfjörð Bjarmason] +- Fixed typo in doc: tls_1_2 => tls1_2. [Maximilian Kaul] +- IMAP: UIDPLUS: correctly warn about weird responses from some servers. [Nicolas Sebrecht] +- website-doc: force copy of the new HTML generated man pages. [Nicolas Sebrecht] +- Makefile: fix clean target. [Nicolas Sebrecht] + +#### Changes + +- MAINTAINERS: benutzer193 joins the testers team. [Nicolas Sebrecht] +- IMAP: UIDPLUS: improve error message on response error for new UID. [Nicolas Sebrecht] +- Display the imaplib and python versions for each normal run. [Nicolas Sebrecht] +- imapserver: provide some SSL info while in imap debug mode. [Nicolas Sebrecht] +- manual: improve the documentation about sqlite migration. [Nicolas Sebrecht] +- documentation: add entry for faulting folders with Microsoft servers. [Nicolas Sebrecht] +- website-doc.sh: add hint on API removal. [Nicolas Sebrecht] +- README: refactorize sections. [Nicolas Sebrecht] + + + +### OfflineIMAP v7.1.0 (2017-04-16) + +#### Notes + +The most important change is the removal of the status_backend configuration +option and that's why we're moving to v7.1.0. + +There are other small bug fixes and improvements. However, the codebase didn't +change much since v7.0.14. + +#### Authors + +- Nicolas Sebrecht (6) +- benutzer193 (4) +- Ilias Tsitsimpis (1) + +#### Fixes + +- doc: Fix typo in offlineimap.1 man page. [Ilias Tsitsimpis] +- README: we moved to imaplib2 v2.57. [Nicolas Sebrecht] +- README: mark porting to py3 as stalled. [Nicolas Sebrecht] +- folder: UIDMaps: ignore KeyError failure while removing keys. [Nicolas Sebrecht] + +#### Changes + +- Remove support for the status_backend configuration option. [Nicolas Sebrecht] +- folder/IMAP: improve handling of "matchinguids" error while searching headers. [Nicolas Sebrecht] +- Adjust README to systemd service file changes. [benutzer193] +- Remove oneshot switch from systemd services. [benutzer193] +- Use oneshot services for systemd timers. [benutzer193] +- Create systemd oneshot services. [benutzer193] +- website-doc.sh: versions.yml: set versions in order. [Nicolas Sebrecht] + + + +### OfflineIMAP v7.0.14 (2017-03-11) + +#### Notes + +Here is a new small fixup release for the v7.0 series. The first v7.0.0 release +is near to 8 months old. This v7.0.14 release is more reliable than v6.7.0.3. +Hence, I'm deprecating the v6.7 series. + +Now, you are all enjoined to migrate to v7.0.14. Migrating back to v6.7 is not +supported so you might like to backup your local maildirs and metadata first. + +We will fully remove the legacy text backend driver in near future. The SQLite +driver proved to be better for both performance and reliability. + +With this release we use imaplib2 v2.57 to support some faulting IMAP servers, +fix a little bug about the backend migration to SQLite and serialize the sync +processes to prevent from issues when both IDLE and autorefresh are enabled. + +Happy sync'ing! + +#### Authors + +- Nicolas Sebrecht (5) +- 927589452 (2) +- Jens Heinrich (1) +- Stéphane Graber (1) + +#### Fixes + +- SQLite: avoid concurrent writes on backend migration. [Nicolas Sebrecht] +- Fix ipv6 configuration handling. [Stéphane Graber] +- Prevent synchronization of identical folders from multiple threads. [Nicolas Sebrecht] + +#### Changes + +- Bump from imaplib2 v2.55 to v2.57. [Nicolas Sebrecht] +- scripts/get-repository.sh: use portable /bin/sh. [Jens Heinrich] +- MAINTAINERS: add new tester. [Nicolas Sebrecht] +- scripts/get-repository.sh: use env to call bash. [mailinglists@927589452.de] + + + +### OfflineIMAP v7.0.13 (2017-01-27) + +#### Notes + +Here is a small release with some new features. IMAP servers are better supported. + +The release cycle was improved. Now, we include a new freeze time before +important releases. + +#### Authors + +- Nicolas Sebrecht (8) +- lkcl (2) +- Chris Smart (1) + +#### Features + +- init: register SIGABRT and handle as per SIGUSR2. [Chris Smart] +- add documentation about SIGABRT. [Nicolas Sebrecht] +- learn repository retrycount configuration option. [lkcl] +- learn authproxy configuration option. [lkcl] + +#### Fixes + +- folder: IMAP: add missing whitespace in error message. [Nicolas Sebrecht] +- repository: IMAP: correctly check the response while listing remote folders. [Nicolas Sebrecht] +- release.sh: correctly sort releases to compute latest stable and rc. [Nicolas Sebrecht] + +#### Changes + +- manual: KNOWN ISSUES: add documentation about the deletions. [Nicolas Sebrecht] +- folder: IMAP: improve error message when Dovecot returns any data for UID FETCH. [Nicolas Sebrecht] +- MAINTAINERS: add new official testers. [Nicolas Sebrecht] + + + ### OfflineIMAP v7.0.12 (2016-11-30) #### Notes diff -Nru offlineimap-7.0.12+dfsg1/contrib/helpers.py offlineimap-7.1.2+dfsg1/contrib/helpers.py --- offlineimap-7.0.12+dfsg1/contrib/helpers.py 1970-01-01 00:00:00.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/helpers.py 2017-07-10 14:58:50.000000000 +0000 @@ -0,0 +1,327 @@ +""" + +Put into Public Domain, by Nicolas Sebrecht. + +Helpers for maintenance scripts. + +""" + +from os import chdir, makedirs, system, getcwd +from os.path import expanduser +import shlex +from subprocess import check_output, check_call, CalledProcessError + +import yaml + + +FS_ENCODING = 'UTF-8' +ENCODING = 'UTF-8' + +MAILING_LIST = 'offlineimap-project@lists.alioth.debian.org' +CACHEDIR = '.git/offlineimap-release' +EDITOR = 'vim' +MAILALIASES_FILE = expanduser('~/.mutt/mail_aliases') +TESTERS_FILE = "{}/testers.yml".format(CACHEDIR) +ME = "Nicolas Sebrecht " + + +def run(cmd): + return check_output(cmd, timeout=5).rstrip() + +def goTo(path): + try: + chdir(path) + return True + except FileNotFoundError: + print("Could not find the '{}' directory in '{}'...".format( + path, getcwd()) + ) + return False + + +class Author(object): + def __init__(self, name, count, email): + self.name = name + self.count = count + self.email = email + + def getName(self): + return self.name + + def getCount(self): + return self.count + + def getEmail(self): + return self.email + + +class Git(object): + @staticmethod + def getShortlog(ref): + shortlog = "" + + cmd = shlex.split("git shortlog --no-merges -n v{}..".format(ref)) + output = run(cmd).decode(ENCODING) + + for line in output.split("\n"): + if len(line) > 0: + if line[0] != " ": + line = " {}\n".format(line) + else: + line = " {}\n".format(line.lstrip()) + else: + line = "\n" + + shortlog += line + + return shortlog + + @staticmethod + def add(files): + cmd = shlex.split("git add -- {}".format(files)) + return run(cmd).decode(ENCODING) + + @staticmethod + def commit(msg): + cmd = shlex.split("git commit -s -m 'v{}'".format(msg)) + return run(cmd).decode(ENCODING) + + @staticmethod + def tag(version): + cmd = shlex.split("git tag -a 'v{}' -m 'v{}'".format(version, version)) + return run(cmd).decode(ENCODING) + + @staticmethod + def stash(msg): + cmd = shlex.split("git stash create '{}'".format(msg)) + return run(cmd).decode(ENCODING) + + @staticmethod + def mergeFF(ref): + cmd = shlex.split("git merge --ff '{}'".format(ref)) + return run(cmd).decode(ENCODING) + + @staticmethod + def getDiffstat(ref): + cmd = shlex.split("git diff --stat v{}..".format(ref)) + return run(cmd).decode(ENCODING) + + @staticmethod + def isClean(): + try: + check_call(shlex.split("git diff --quiet")) + check_call(shlex.split("git diff --cached --quiet")) + except CalledProcessError: + return False + return True + + @staticmethod + def buildMessageId(): + cmd = shlex.split( + "git log HEAD~1.. --oneline --pretty='%H.%t.upcoming.%ce'") + return run(cmd).decode(ENCODING) + + @staticmethod + def resetKeep(ref): + return run(shlex.split("git reset --keep {}".format(ref))) + + @staticmethod + def getRef(ref): + return run(shlex.split("git rev-parse {}".format(ref))).rstrip() + + @staticmethod + def rmTag(tag): + return run(shlex.split("git tag -d {}".format(tag))) + + @staticmethod + def checkout(ref, create=False): + if create: + create = "-b" + else: + create = "" + + cmd = shlex.split("git checkout {} {}".format(create, ref)) + run(cmd) + head = shlex.split("git rev-parse HEAD") + revparseRef = shlex.split("git rev-parse {}".format(ref)) + if run(head) != run(revparseRef): + raise Exception("checkout to '{}' did not work".format(ref)) + + @staticmethod + def makeCacheDir(): + try: + makedirs(CACHEDIR) + except FileExistsError: + pass + + @staticmethod + def getLocalUser(): + cmd = shlex.split("git config --get user.name") + name = run(cmd).decode(ENCODING) + cmd = shlex.split("git config --get user.email") + email = run(cmd).decode(ENCODING) + return name, email + + @staticmethod + def buildDate(): + cmd = shlex.split("git log HEAD~1.. --oneline --pretty='%cD'") + return run(cmd).decode(ENCODING) + + @staticmethod + def getAuthorsList(sinceRef): + authors = [] + + cmd = shlex.split("git shortlog --no-merges -sne v{}..".format(sinceRef)) + output = run(cmd).decode(ENCODING) + + for line in output.split("\n"): + count, full = line.strip().split("\t") + full = full.split(' ') + name = ' '.join(full[:-1]) + email = full[-1] + + authors.append(Author(name, count, email)) + + return authors + + @staticmethod + def getCommitsList(sinceRef): + cmd = shlex.split( + "git log --no-merges --format='- %h %s. [%aN]' v{}..".format(sinceRef) + ) + return run(cmd).decode(ENCODING) + + @staticmethod + def chdirToRepositoryTopLevel(): + cmd = shlex.split("git rev-parse --show-toplevel") + topLevel = run(cmd) + + chdir(topLevel) + + +class OfflineimapInfo(object): + def getVersion(self): + cmd = shlex.split("./offlineimap.py --version") + return run(cmd).rstrip().decode(FS_ENCODING) + + def editInit(self): + return system("{} ./offlineimap/__init__.py".format(EDITOR)) + + + +class User(object): + """Interact with the user.""" + + @staticmethod + def request(msg, prompt='--> '): + print(msg) + return input(prompt) + + @staticmethod + def pause(msg=False): + return User.request(msg, prompt="Press Enter to continue..") + + @staticmethod + def yesNo(msg, defaultToYes=False, prompt='--> '): + endMsg = " [y/N]: No" + if defaultToYes: + endMsg = " [Y/n]: Yes" + msg += endMsg + answer = User.request(msg, prompt).lower() + if answer in ['y', 'yes']: + return True + if defaultToYes and answer not in ['n', 'no']: + return True + return False + + +class Tester(object): + def __init__(self, name, email, feedback): + self.name = name + self.email = email + self.feedback = feedback + + def __str__(self): + return "{} {}".format(self.name, self.email) + + def getName(self): + return self.name + + def getEmail(self): + return self.email + + def getFeedback(self): + return self.feedback + + def positiveFeedback(self): + return self.feedback is True + + def setFeedback(self, feedback): + assert feedback in [True, False, None] + self.feedback = feedback + + def switchFeedback(self): + self.feedback = not self.feedback + + +class Testers(object): + def __init__(self): + self.testers = None + self._read() + + def _read(self): + self.testers = [] + with open(TESTERS_FILE, 'r') as fd: + testers = yaml.load(fd) + for tester in testers: + name = tester['name'] + email = tester['email'] + feedback = tester['feedback'] + self.testers.append(Tester(name, email, feedback)) + + @staticmethod + def listTestersInTeam(): + """Returns a list of emails extracted from my mailaliases file.""" + + cmd = shlex.split("grep offlineimap-testers {}".format(MAILALIASES_FILE)) + output = run(cmd).decode(ENCODING) + emails = output.lstrip("alias offlineimap-testers ").split(', ') + return emails + + def add(self, name, email, feedback=None): + self.testers.append(Tester(name, email, feedback)) + + def remove(self, tester): + self.testers.remove(tester) + + def get(self): + return self.testers + + def getList(self): + testersList = "" + for tester in self.testers: + testersList += "- {}\n".format(tester.getName()) + return testersList + + def getListOk(self): + testersOk = [] + for tester in self.testers: + if tester.positiveFeedback(): + testersOk.append(tester) + return testersOk + + def reset(self): + for tester in self.testers: + tester.setFeedback(None) + + def write(self): + testers = [] + for tester in self.testers: + testers.append({ + 'name': tester.getName(), + 'email': tester.getEmail(), + 'feedback': tester.getFeedback(), + }) + with open(TESTERS_FILE, 'w') as fd: + fd.write(yaml.dump(testers)) + diff -Nru offlineimap-7.0.12+dfsg1/contrib/internet-urllib3.py offlineimap-7.1.2+dfsg1/contrib/internet-urllib3.py --- offlineimap-7.0.12+dfsg1/contrib/internet-urllib3.py 1970-01-01 00:00:00.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/internet-urllib3.py 2017-07-10 14:58:50.000000000 +0000 @@ -0,0 +1,19 @@ +#!/usr/bin/env python + +import urllib3 +import certifi + +def isInternetConnected(url="www.ietf.org"): + result = False + http = urllib3.PoolManager( + cert_reqs='CERT_REQUIRED', # Force certificate check. + ca_certs=certifi.where(), # Path to the Certifi bundle. + ) + try: + r = http.request('HEAD', 'https://' + url) + result = True + except Exception as e: # urllib3.exceptions.SSLError + result = False + return result + +print isInternetConnected() diff -Nru offlineimap-7.0.12+dfsg1/contrib/release.py offlineimap-7.1.2+dfsg1/contrib/release.py --- offlineimap-7.0.12+dfsg1/contrib/release.py 1970-01-01 00:00:00.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/release.py 2017-07-10 14:58:50.000000000 +0000 @@ -0,0 +1,460 @@ +#!/usr/bin/python3 + +""" + +Put into Public Domain, by Nicolas Sebrecht. + +Make a new release. + +""" + +#TODO: announce: cc list on announce includes all testers +#TODO: announce: remove empty sections +#TODO: websitedoc up +#TODO: website branch not including all changes! + + +from os import system, path, rename +from datetime import datetime +from subprocess import check_call +import shlex +import time +from email import utils + +from helpers import ( + MAILING_LIST, CACHEDIR, EDITOR, Git, OfflineimapInfo, Testers, User, run, goTo +) + + +__VERSION__ = "0.1" + +SPHINXBUILD = 'sphinx-build' +DOCSDIR = 'docs' +CHANGELOG_MAGIC = '{:toc}' +WEBSITE_LATEST = "website/_data/latest.yml" + +CHANGELOG_EXCERPT = "{}/changelog.excerpt.md".format(CACHEDIR) +CHANGELOG_EXCERPT_OLD = "{}.old".format(CHANGELOG_EXCERPT) +CHANGELOG = "Changelog.md" +ANNOUNCE_FILE = "{}/announce.txt".format(CACHEDIR) + +WEBSITE_LATEST_SKEL = """# DO NOT EDIT MANUALLY: it is generated by the release script. +stable: v{stable} +""" + +CHANGELOG_SKEL = """ +### OfflineIMAP v{version} ({date}) + +#### Notes + + +This release was tested by: + +{testersList} + +#### Authors + +{authorsList} + +#### Features + + +#### Fixes + + +#### Changes + + + +{commitsList} + +""" + +END_MESSAGE = """ +Release is ready! +Make your checks and push the changes for both offlineimap and the website. +Announce template stands in '{announce}'. +Command samples to do manually: + +- git push master next {new_version} +- python setup.py sdist && twine upload dist/* && rm -rf dist MANIFEST +- cd website +- git checkout master +- git merge {website_branch} +- git push master +- cd .. +- git send-email {announce} + +...and write a Twitter message. +Have fun! ,-) +""" + + +class State(object): + def __init__(self): + self.master = None + self.next = None + self.website = None + self.tag = None + + def setTag(self, tag): + self.tag = tag + + def save(self): + self.master = Git.getRef('master') + self.next = Git.getRef('next') + + def saveWebsite(self): + Git.chdirToRepositoryTopLevel() + goTo('website') + self.website = Git.getRef('master') + goTo('..') + + def restore(self): + Git.chdirToRepositoryTopLevel() + try: + Git.checkout('-f') + except: + pass + # Git.checkout('master') + # Git.resetKeep(self.master) + # Git.checkout('next') + # Git.resetKeep(self.next) + + if self.tag is not None: + Git.rmTag(self.tag) + + if self.website is not None: + if goTo('website'): + Git.checkout(self.website) + goTo('..') + + +class Changelog(object): + def __init__(self): + self.shouldUsePrevious = False + + def edit(self): + return system("{} {}".format(EDITOR, CHANGELOG_EXCERPT)) + + def update(self): + # Insert excerpt to CHANGELOG. + system("sed -i -e '/{}/ r {}' '{}'".format( + CHANGELOG_MAGIC, CHANGELOG_EXCERPT, CHANGELOG + ) + ) + # Remove trailing whitespaces. + system("sed -i -r -e 's, +$,,' '{}'".format(CHANGELOG)) + + def savePrevious(self): + rename(CHANGELOG_EXCERPT, CHANGELOG_EXCERPT_OLD) + + def isPrevious(self): + if path.isfile(CHANGELOG_EXCERPT_OLD): + return True + return False + + def showPrevious(self): + output = run(shlex.split("cat '{}'".format(CHANGELOG_EXCERPT_OLD))) + for line in output.splitlines(): + print(line.decode('utf-8')) # Weird to have to decode bytes here. + + def usePrevious(self): + rename(CHANGELOG_EXCERPT_OLD, CHANGELOG_EXCERPT) + self.shouldUsePrevious = True + + def usingPrevious(self): + return self.shouldUsePrevious + + def writeExcerpt(self, version, date, + testersList, authorsList, commitsList): + + with open(CHANGELOG_EXCERPT, 'w+') as fd: + fd.write(CHANGELOG_SKEL.format( + version=version, + date=date, + testersList=testersList, + authorsList=authorsList, + commitsList=commitsList, + )) + + def getSectionsContent(self): + dict_Content = {} + + with open(CHANGELOG_EXCERPT, 'r') as fd: + currentSection = None + for line in fd: + line = line.rstrip() + if line == "#### Notes": + currentSection = 'Notes' + dict_Content['Notes'] = "" + continue # Don't keep this title. + elif line == "#### Authors": + currentSection = 'Authors' + dict_Content['Authors'] = "" + continue # Don't keep this title. + elif line == "#### Features": + currentSection = 'Features' + dict_Content['Features'] = "" + continue # Don't keep this title. + elif line == "#### Fixes": + currentSection = 'Fixes' + dict_Content['Fixes'] = "" + continue # Don't keep this title. + elif line == "#### Changes": + currentSection = 'Changes' + dict_Content['Changes'] = "" + continue # Don't keep this title. + elif line == "-- ": + break # Stop extraction. + + if currentSection is not None: + dict_Content[currentSection] += "{}\n".format(line) + + #TODO: cleanup empty sections. + return dict_Content + + +class Announce(object): + def __init__(self, version): + self.fd = open(ANNOUNCE_FILE, 'w') + self.version = version + + def setHeaders(self, messageId, date): + self.fd.write("Message-Id: <{}>\n".format(messageId)) + self.fd.write("Date: {}\n".format(date)) + self.fd.write("From: Nicolas Sebrecht \n") + self.fd.write("To: {}\n".format(MAILING_LIST)) + self.fd.write( + "Subject: [ANNOUNCE] OfflineIMAP v{} released\n".format(self.version)) + self.fd.write("\n") + + self.fd.write(""" +OfflineIMAP v{version} is out. + +Downloads: + http://github.com/OfflineIMAP/offlineimap/archive/v{version}.tar.gz + http://github.com/OfflineIMAP/offlineimap/archive/v{version}.zip + +Pip: + wget "https://raw.githubusercontent.com/OfflineIMAP/offlineimap/v{version}/requirements.txt" -O requirements.txt + pip install -r ./requirements.txt --user git+https://github.com/OfflineIMAP/offlineimap.git@v{version} + +""".format(version=self.version) + ) + + def setContent(self, dict_Content): + self.fd.write("\n") + for section in ['Notes', 'Authors', 'Features', 'Fixes', 'Changes']: + if section in dict_Content: + if section != "Notes": + self.fd.write("# {}\n".format(section)) + self.fd.write(dict_Content[section]) + self.fd.write("\n") + # Signature. + self.fd.write("-- \n") + self.fd.write("Nicolas Sebrecht\n") + + def close(self): + self.fd.close() + + +class Website(object): + def updateAPI(self): + req = "update API of the website? (requires {}) [Y/n]".format(SPHINXBUILD) + if not User.yesNo(req, defaultToYes=True): + return False + + if check_call(shlex.split("{} --version".format(SPHINXBUILD))) != 0: + print(""" +Oops! you don't have {} installed?" +Cannot update the webite documentation..." +You should install it and manually run:" + $ cd {}" + $ make websitedoc" +Then, commit and push changes of the website.""".format(SPHINXBUILD, DOCSDIR)) + User.pause() + return False + + Git.chdirToRepositoryTopLevel() + if not goTo('website'): + User.pause() + return False + if not Git.isClean: + print("There is WIP in the website repository: stashing") + Git.stash('WIP during offlineimap API import') + + goTo('..') + return True + + def buildLatest(self, version): + Git.chdirToRepositoryTopLevel() + with open(WEBSITE_LATEST, 'w') as fd: + fd.write(WEBSITE_LATEST_SKEL.format(stable=version)) + + def exportDocs(self, version): + branchName = "import-v{}".format(version) + + if not goTo(DOCSDIR): + User.pause() + return + + if check_call(shlex.split("make websitedoc")) != 0: + print("error while calling 'make websitedoc'") + exit(3) + + Git.chdirToRepositoryTopLevel() + if not goTo("website"): + User.pause() + return + + Git.checkout(branchName, create=True) + Git.add('_doc/versions') + Git.commit("update for offlineimap v{}".format(version)) + + User.pause( + "website: branch '{}' is ready for a merge in master!".format( + branchName + ) + ) + goTo('..') + return branchName + + +class Release(object): + def __init__(self): + self.state = State() + self.offlineimapInfo = OfflineimapInfo() + self.testers = Testers() + self.changelog = Changelog() + self.websiteBranch = "NO_BRANCH_NAME_ERROR" + + + def getVersion(self): + return self.offlineimapInfo.getVersion() + + def prepare(self): + if not Git.isClean(): + print("The git repository is not clean; aborting") + exit(1) + Git.makeCacheDir() + Git.checkout('next') + + def requestVersion(self, currentVersion): + User.request("going to make a new release after {}".format(currentVersion)) + + def updateVersion(self): + self.offlineimapInfo.editInit() + + def checkVersions(self, current, new): + if new == current: + print("version was not changed; stopping.") + exit(1) + + def updateChangelog(self): + if self.changelog.isPrevious(): + self.changelog.showPrevious() + if User.yesNo("A previous Changelog excerpt was found. Use it?"): + self.changelog.usePrevious() + + if not self.changelog.usingPrevious(): + date = datetime.now().strftime('%Y-%m-%d') + testersList = "" + testers = self.testers.getListOk() + authorsList = "" + authors = Git.getAuthorsList(currentVersion) + + for tester in testers: + testersList += "- {}\n".format(tester.getName()) + for author in authors: + authorsList += "- {} ({})\n".format( + author.getName(), author.getCount() + ) + commitsList = Git.getCommitsList(currentVersion) + date = datetime.now().strftime('%Y-%m-%d') + self.changelog.writeExcerpt( + newVersion, date, testersList, authorsList, commitsList + ) + + self.changelog.edit() + self.changelog.update() + + def writeAnnounce(self): + announce = Announce(newVersion) + + messageId = utils.make_msgid('release.py', 'laposte.net') + nowtuple = datetime.now().timetuple() + nowtimestamp = time.mktime(nowtuple) + date = utils.formatdate(nowtimestamp) + + announce.setHeaders(messageId, date) + announce.setContent(self.changelog.getSectionsContent()) + announce.close() + + def make(self): + Git.add('offlineimap/__init__.py') + Git.add('Changelog.md') + commitMsg = "{}\n".format(newVersion) + for tester in self.testers.getListOk(): + commitMsg = "{}\nTested-by: {} {}".format( + commitMsg, tester.getName(), tester.getEmail() + ) + Git.commit(commitMsg) + self.state.setTag(newVersion) + Git.tag(newVersion) + Git.checkout('master') + Git.mergeFF('next') + Git.checkout('next') + + def updateWebsite(self, newVersion): + self.state.saveWebsite() + website = Website() + website.buildLatest(newVersion) + if website.updateAPI(): + self.websiteBranch = website.exportDocs(newVersion) + + def getWebsiteBranch(self): + return self.websiteBranch + + def after(self): + for protectedRun in [self.testers.reset, self.changelog.savePrevious]: + try: + protectedRun() + except Exception as e: + print(e) + + def restore(self): + self.state.restore() + + +if __name__ == '__main__': + release = Release() + Git.chdirToRepositoryTopLevel() + + try: + release.prepare() + currentVersion = release.getVersion() + + release.requestVersion(currentVersion) + release.updateVersion() + newVersion = release.getVersion() + + release.checkVersions(currentVersion, newVersion) + release.updateChangelog() + + release.writeAnnounce() + User.pause() + + release.make() + release.updateWebsite(newVersion) + release.after() + + websiteBranch = release.getWebsiteBranch() + print(END_MESSAGE.format( + announce=ANNOUNCE_FILE, + new_version=newVersion, + website_branch=websiteBranch) + ) + except Exception as e: + release.restore() + raise diff -Nru offlineimap-7.0.12+dfsg1/contrib/release.sh offlineimap-7.1.2+dfsg1/contrib/release.sh --- offlineimap-7.0.12+dfsg1/contrib/release.sh 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/release.sh 2017-07-10 14:58:50.000000000 +0000 @@ -289,11 +289,11 @@ function get_last_rc () { - git tag | grep -E '^v([0-9][\.-]){3}rc' | sort -n | tail -n1 + git tag | grep -E '^v([0-9][\.-]){3}rc' | sort -V | tail -n1 } function get_last_stable () { - git tag | grep -E '^v([0-9][\.])+' | grep -v '\-rc' | sort -n | tail -n1 + git tag | grep -E '^v([0-9][\.])+' | grep -v '\-rc' | sort -V | tail -n1 } function update_website_releases_info() { diff -Nru offlineimap-7.0.12+dfsg1/contrib/systemd/mail.target offlineimap-7.1.2+dfsg1/contrib/systemd/mail.target --- offlineimap-7.0.12+dfsg1/contrib/systemd/mail.target 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/systemd/mail.target 1970-01-01 00:00:00.000000000 +0000 @@ -1,5 +0,0 @@ -[Unit] -Description=Mail Target - -[Install] -WantedBy=default.target diff -Nru offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap-oneshot.service offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap-oneshot.service --- offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap-oneshot.service 1970-01-01 00:00:00.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap-oneshot.service 2017-07-10 14:58:50.000000000 +0000 @@ -0,0 +1,12 @@ +[Unit] +Description=Offlineimap Service (oneshot) +Documentation=man:offlineimap(1) + +[Service] +Type=oneshot +ExecStart=/usr/bin/offlineimap -o -u basic +# Give 120 seconds for offlineimap to gracefully stop before hard killing it: +TimeoutStopSec=120 + +[Install] +WantedBy=mail.target diff -Nru offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap-oneshot@.service offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap-oneshot@.service --- offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap-oneshot@.service 1970-01-01 00:00:00.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap-oneshot@.service 2017-07-10 14:58:50.000000000 +0000 @@ -0,0 +1,12 @@ +[Unit] +Description=Offlineimap Service for account %i (oneshot) +Documentation=man:offlineimap(1) + +[Service] +Type=oneshot +ExecStart=/usr/bin/offlineimap -o -a %i -u basic +# Give 120 seconds for offlineimap to gracefully stop before hard killing it. +TimeoutStopSec=120 + +[Install] +WantedBy=default.target diff -Nru offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap-oneshot.timer offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap-oneshot.timer --- offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap-oneshot.timer 1970-01-01 00:00:00.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap-oneshot.timer 2017-07-10 14:58:50.000000000 +0000 @@ -0,0 +1,9 @@ +[Unit] +Description=Offlineimap Query Timer + +[Timer] +OnBootSec=1m +OnUnitInactiveSec=15m + +[Install] +WantedBy=default.target diff -Nru offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap-oneshot@.timer offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap-oneshot@.timer --- offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap-oneshot@.timer 1970-01-01 00:00:00.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap-oneshot@.timer 2017-07-10 14:58:50.000000000 +0000 @@ -0,0 +1,9 @@ +[Unit] +Description=Offlineimap Query Timer for account %i + +[Timer] +OnBootSec=1m +OnUnitInactiveSec=15m + +[Install] +WantedBy=default.target diff -Nru offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap.service offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap.service --- offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap.service 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap.service 2017-07-10 14:58:50.000000000 +0000 @@ -1,11 +1,11 @@ [Unit] Description=Offlineimap Service +Documentation=man:offlineimap(1) [Service] -Type=oneshot -ExecStart=/usr/bin/offlineimap -o -u syslog -# Give 12 seconds for offlineimap to gracefully stop before hard killing it. -TimeoutStopSec=12 +ExecStart=/usr/bin/offlineimap -u basic +Restart=on-failure +RestartSec=60 [Install] -WantedBy=mail.target +WantedBy=default.target diff -Nru offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap@.service offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap@.service --- offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap@.service 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap@.service 2017-07-10 14:58:50.000000000 +0000 @@ -1,9 +1,11 @@ [Unit] Description=Offlineimap Service for account %i +Documentation=man:offlineimap(1) [Service] -Type=oneshot -ExecStart=/usr/bin/offlineimap -o -a %i -u syslog +ExecStart=/usr/bin/offlineimap -a %i -u basic +Restart=on-failure +RestartSec=60 [Install] -WantedBy=mail.target +WantedBy=default.target diff -Nru offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap.timer offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap.timer --- offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap.timer 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap.timer 1970-01-01 00:00:00.000000000 +0000 @@ -1,9 +0,0 @@ -[Unit] -Description=Offlineimap Query Timer - -[Timer] -OnUnitInactiveSec=15m -Unit=offlineimap.service - -[Install] -WantedBy=mail.target diff -Nru offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap@.timer offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap@.timer --- offlineimap-7.0.12+dfsg1/contrib/systemd/offlineimap@.timer 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/systemd/offlineimap@.timer 1970-01-01 00:00:00.000000000 +0000 @@ -1,9 +0,0 @@ -[Unit] -Description=Offlineimap Query Timer for account %i - -[Timer] -OnUnitInactiveSec=15m -Unit=offlineimap@%i.service - -[Install] -WantedBy=mail.target diff -Nru offlineimap-7.0.12+dfsg1/contrib/systemd/README.md offlineimap-7.1.2+dfsg1/contrib/systemd/README.md --- offlineimap-7.0.12+dfsg1/contrib/systemd/README.md 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/systemd/README.md 2017-07-10 14:58:50.000000000 +0000 @@ -3,8 +3,8 @@ title: Integrating OfflineIMAP into systemd author: Ben Boeckel date: 2015-03-22 -contributors: Abdo Roig-Maranges -updated: 2015-03-25 +contributors: Abdo Roig-Maranges, benutzer193, Hugo Osvaldo Barrera +updated: 2017-06-01 --- @@ -12,8 +12,16 @@ ## Systemd units -These unit files are meant to be used in the user session. You may drop them into `/etc/systemd/user` or `${XDG_DATA_HOME}/systemd/user` followed by `systemctl --user daemon-reload` to have systemd aware of the unit files. - -These files are meant to be triggered either manually using `systemctl --user start offlineimap.service` or by enabling the timer unit using `systemctl --user enable offlineimap.timer`. Additionally, specific accounts may be triggered by using `offlineimap@myaccount.timer` or `offlineimap@myaccount.service`. - -These unit files are installed as being enabled via a `mail.target` unit which is intended to be a catch-all for mail-related unit files. A simple `mail.target` file is also provided. +These unit files are meant to be used in the user session. You may drop them +into `/etc/systemd/user` or `${XDG_DATA_HOME}/systemd/user` followed by +`systemctl --user daemon-reload` to have systemd aware of the unit files. + +These files are meant to be triggered either manually using `systemctl --user +start offlineimap.service` or by enabling the timer unit using `systemctl --user +enable offlineimap-oneshot.timer`. Additionally, specific accounts may be +triggered by using `offlineimap@myaccount.timer` or +`offlineimap-oneshot@myaccount.service`. + +If the defaults provided by these units doesn't suit your setup, any of the +values may be overridden by using `systemctl --user edit offlineimap.service`. +This'll prevent having to copy-and-edit the original file. diff -Nru offlineimap-7.0.12+dfsg1/contrib/tested-by.py offlineimap-7.1.2+dfsg1/contrib/tested-by.py --- offlineimap-7.0.12+dfsg1/contrib/tested-by.py 1970-01-01 00:00:00.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/tested-by.py 2017-07-10 14:58:50.000000000 +0000 @@ -0,0 +1,142 @@ +#!/usr/bin/python3 + +""" + +Put into Public Domain, by Nicolas Sebrecht. + +Manage the feedbacks of the testers for the release notes. + +""" + +from os import system +import argparse + +from helpers import CACHEDIR, EDITOR, Testers, User, Git + + +class App(object): + def __init__(self): + self.args = None + self.testers = Testers() + self.feedbacks = None + + + def _getTestersByFeedback(self): + if self.feedbacks is not None: + return self.feedbacks + + feedbackOk = [] + feedbackNo = [] + + for tester in self.testers.get(): + if tester.getFeedback() is True: + feedbackOk.append(tester) + else: + feedbackNo.append(tester) + + for array in [feedbackOk, feedbackNo]: + array.sort(key=lambda t: t.getName()) + + self.feedbacks = feedbackOk + feedbackNo + + def parseArgs(self): + parser = argparse.ArgumentParser(description='Manage the feedbacks.') + + parser.add_argument('--add', '-a', dest='add_tester', + help='Add tester') + parser.add_argument('--delete', '-d', dest='delete_tester', + type=int, + help='Delete tester NUMBER') + parser.add_argument('--list', '-l', dest='list_all_testers', + action='store_true', + help='List the testers') + parser.add_argument('--switchFeedback', '-s', dest='switch_feedback', + action='store_true', + help='Switch the feedback of a tester') + + self.args = parser.parse_args() + + def run(self): + if self.args.list_all_testers is True: + self.listTesters() + if self.args.switch_feedback is True: + self.switchFeedback() + elif self.args.add_tester: + self.addTester(self.args.add_tester) + elif type(self.args.delete_tester) == int: + self.deleteTester(self.args.delete_tester) + + def addTester(self, strTester): + try: + splitted = strTester.split('<') + name = splitted[0].strip() + email = "<{}".format(splitted[1]).strip() + except Exception as e: + print(e) + print("expected format is: 'Firstname Lastname '") + exit(2) + self.testers.add(name, email) + self.testers.write() + + def deleteTester(self, number): + self.listTesters() + removed = self.feedbacks.pop(number) + self.testers.remove(removed) + + print("New list:") + self.feedbacks = None + self.listTesters() + print("Removed: {}".format(removed)) + ans = User.request("Save on disk? (s/Q)").lower() + if ans in ['s']: + self.testers.write() + + + def listTesters(self): + self._getTestersByFeedback() + + count = 0 + for tester in self.feedbacks: + feedback = "ok" + if tester.getFeedback() is not True: + feedback = "no" + print("{:02d} - {} {}: {}".format( + count, tester.getName(), tester.getEmail(), feedback + ) + ) + count += 1 + + def switchFeedback(self): + self._getTestersByFeedback() + msg = "Switch tester: [/s/q]" + + self.listTesters() + number = User.request(msg) + while number.lower() not in ['s', 'save', 'q', 'quit']: + if number == '': + continue + try: + number = int(number) + self.feedbacks[number].switchFeedback() + except (ValueError, IndexError) as e: + print(e) + exit(1) + finally: + self.listTesters() + number = User.request(msg) + if number in ['s', 'save']: + self.testers.write() + self.listTesters() + + def reset(self): + self.testers.reset() + self.testers.write() + + #def updateMailaliases(self): + +if __name__ == '__main__': + Git.chdirToRepositoryTopLevel() + + app = App() + app.parseArgs() + app.run() diff -Nru offlineimap-7.0.12+dfsg1/contrib/upcoming.py offlineimap-7.1.2+dfsg1/contrib/upcoming.py --- offlineimap-7.0.12+dfsg1/contrib/upcoming.py 1970-01-01 00:00:00.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/contrib/upcoming.py 2017-07-10 14:58:50.000000000 +0000 @@ -0,0 +1,74 @@ +#!/usr/bin/python3 + +""" + +Put into Public Domain, by Nicolas Sebrecht. + +Produce the "upcoming release" notes. + +""" + +from os import system + +from helpers import ( + MAILING_LIST, CACHEDIR, EDITOR, Testers, Git, OfflineimapInfo, User +) + + + +UPCOMING_FILE = "{}/upcoming.txt".format(CACHEDIR) + +UPCOMING_HEADER = """ +Message-Id: <{messageId}> +Date: {date} +From: {name} <{email}> +To: {mailinglist} +Cc: {ccList} +Subject: [ANNOUNCE] upcoming offlineimap v{expectedVersion} + +# Notes + +I think it's time for a new release. + +I aim to make the new release in one week, approximately. If you'd like more +time, please let me know. ,-) + +Please, send me a mail to confirm it works for you. This will be written in the +release notes and the git logs. + + +# Authors + +""" + + +if __name__ == '__main__': + offlineimapInfo = OfflineimapInfo() + + Git.chdirToRepositoryTopLevel() + oVersion = offlineimapInfo.getVersion() + ccList = Testers.listTestersInTeam() + authors = Git.getAuthorsList(oVersion) + for author in authors: + email = author.getEmail() + if email not in ccList: + ccList.append(email) + + with open(UPCOMING_FILE, 'w') as upcoming: + header = {} + + header['messageId'] = Git.buildMessageId() + header['date'] = Git.buildDate() + header['name'], header['email'] = Git.getLocalUser() + header['mailinglist'] = MAILING_LIST + header['expectedVersion'] = User.request("Expected new version?") + header['ccList'] = ", ".join(ccList) + + upcoming.write(UPCOMING_HEADER.format(**header).lstrip()) + upcoming.write(Git.getShortlog(oVersion)) + + upcoming.write("\n\n# Diffstat\n\n") + upcoming.write(Git.getDiffstat(oVersion)) + upcoming.write("\n\n\n-- \n{}\n".format(Git.getLocalUser()[0])) + + system("{} {}".format(EDITOR, UPCOMING_FILE)) diff -Nru offlineimap-7.0.12+dfsg1/debian/changelog offlineimap-7.1.2+dfsg1/debian/changelog --- offlineimap-7.0.12+dfsg1/debian/changelog 2017-05-03 15:40:38.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/changelog 2017-07-11 15:00:33.000000000 +0000 @@ -1,3 +1,35 @@ +offlineimap (7.1.2+dfsg1-1) unstable; urgency=low + + * New upstream release + * Propagate changes from experimental (7.0.14+dfsg1-1) to unstable + * Update the `strip-nondeterminism/a2x` script to use NO_FAKE_STAT=1. + Also, newer versions of debhelper (versions >= 9.20151004) export the + SOURCE_DATE_EPOCH variable during builds, so update d/rules accordingly. + * Remove Dimitri John Ledkov from uploaders. + Thanks for maintaining offlineimap. (Closes: #867809) + * Update my email address + * Bump Standards-Version to 4.0.0, no changes needed + * Remove obsolete d/clean file. + All the build artifacts are now being removed by `make clean`. + * Update d/copyright file (add new files) + * Document the removal of the status_backend configuration option in d/NEWS. + Thanks to Henrique de Moraes Holschuh for suggesting eatmydata. + * Update d/patches + - Drop Acquire-lock-before-updating-the-CursesLogHandler-window (applied + upstream) + - Update patches to apply cleanly on new upstream + + -- Ilias Tsitsimpis Tue, 11 Jul 2017 18:00:33 +0300 + +offlineimap (7.0.14+dfsg1-1) experimental; urgency=medium + + * New upstream release. + * Update d/copyright file (bump Copyright years). + * Bump debhelper compat level to 10. + * Bump versioned dependency of debhelper (>= 10). + + -- Ilias Tsitsimpis Thu, 23 Mar 2017 12:51:38 +0200 + offlineimap (7.0.12+dfsg1-2) unstable; urgency=medium * Backport upstream patch to fix Blinkenlights UI. diff -Nru offlineimap-7.0.12+dfsg1/debian/clean offlineimap-7.1.2+dfsg1/debian/clean --- offlineimap-7.0.12+dfsg1/debian/clean 2017-03-24 16:46:38.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/clean 1970-01-01 00:00:00.000000000 +0000 @@ -1,2 +0,0 @@ -docs/offlineimap.1 -docs/offlineimapui.7 diff -Nru offlineimap-7.0.12+dfsg1/debian/compat offlineimap-7.1.2+dfsg1/debian/compat --- offlineimap-7.0.12+dfsg1/debian/compat 2017-05-03 15:18:18.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/compat 2017-07-11 13:58:13.000000000 +0000 @@ -1 +1 @@ -9 +10 diff -Nru offlineimap-7.0.12+dfsg1/debian/control offlineimap-7.1.2+dfsg1/debian/control --- offlineimap-7.0.12+dfsg1/debian/control 2017-05-03 15:18:18.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/control 2017-07-11 15:00:33.000000000 +0000 @@ -1,10 +1,9 @@ Source: offlineimap Section: mail Priority: optional -Maintainer: Ilias Tsitsimpis -Uploaders: Dimitri John Ledkov +Maintainer: Ilias Tsitsimpis Build-Depends: - debhelper (>= 9), + debhelper (>= 10), python, dh-python, python-setuptools, @@ -17,7 +16,7 @@ docbook-xsl, libxml2-utils X-Python-Version: >= 2.7 -Standards-Version: 3.9.8 +Standards-Version: 4.0.0 Homepage: http://offlineimap.org/ Vcs-Git: https://anonscm.debian.org/git/collab-maint/offlineimap.git Vcs-Browser: https://anonscm.debian.org/cgit/collab-maint/offlineimap.git diff -Nru offlineimap-7.0.12+dfsg1/debian/copyright offlineimap-7.1.2+dfsg1/debian/copyright --- offlineimap-7.0.12+dfsg1/debian/copyright 2017-05-03 15:18:18.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/copyright 2017-07-11 15:00:33.000000000 +0000 @@ -7,21 +7,22 @@ Files: * Copyright: 2016 Nicolas Sebrecht & contributors - 2002-2016, John Goerzen & contributors + 2002-2017, John Goerzen & contributors 2008, Riccardo Murri - 2009-2016, Stewart Smith and contributors + 2009-2017, Stewart Smith and contributors 2013-2016, Eygene A. Ryabinkin and contributors 2010-2015, Sebastian Spaeth & contributors License: GPL-2+ with OpenSSL exception -Files: scripts/get-repository.sh +Files: scripts/get-repository.sh contrib/helpers.py + contrib/release.py contrib/upcoming.py Copyright: Public Domain License: public-domain This file is in the public domain. Files: debian/* Copyright: 2015, Dimitri John Ledkov - 2015-2016, Ilias Tsitsimpis + 2015-2017, Ilias Tsitsimpis License: GPL-2+ with OpenSSL exception License: GPL-2+ with OpenSSL exception diff -Nru offlineimap-7.0.12+dfsg1/debian/NEWS offlineimap-7.1.2+dfsg1/debian/NEWS --- offlineimap-7.0.12+dfsg1/debian/NEWS 2017-03-24 16:46:38.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/NEWS 2017-07-11 15:00:33.000000000 +0000 @@ -1,3 +1,15 @@ +offlineimap (7.1.2+dfsg1-1) unstable; urgency=low + + The status_backend configuration option has been removed, hence it is no + longer possible to select the (legacy) plain text database. Anyone still + using the old database will automatically migrate to the sqlite one. + + Note that, due to limitations in the code, the initial conversion may be + extremely slow. As a workaround, consider running offlineimap with eatmydata + while converting the database. + + -- Ilias Tsitsimpis Tue, 11 Jul 2017 18:00:33 +0300 + offlineimap (7.0.6+dfsg1-1) unstable; urgency=medium With OfflineIMAP version v7.0.0, the sqlite database becomes the default. diff -Nru offlineimap-7.0.12+dfsg1/debian/patches/Acquire-lock-before-updating-the-CursesLogHandler-window.patch offlineimap-7.1.2+dfsg1/debian/patches/Acquire-lock-before-updating-the-CursesLogHandler-window.patch --- offlineimap-7.0.12+dfsg1/debian/patches/Acquire-lock-before-updating-the-CursesLogHandler-window.patch 2017-05-03 15:29:54.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/patches/Acquire-lock-before-updating-the-CursesLogHandler-window.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,60 +0,0 @@ -From: Ilias Tsitsimpis -Date: Wed, 12 Apr 2017 13:25:46 +0300 -Subject: Acquire lock before updating the CursesLogHandler window - -Make sure that we refresh the screen atomically, since the emit() -function may be called by more that one threads at a time. - -Also, modify the draw_bannerwin() method which used to fail in case the -window would become too small. Make sure that the provided offsets to -the window.addstr() method are properly bounded. - -Closes #160: blinkenlights display is broken -Tested-by: Cyril Brulebois -Tested-by: Gaudenz Steinlin -Signed-off-by: Ilias Tsitsimpis -Signed-off-by: Nicolas Sebrecht - -Origin: upstream, https://github.com/OfflineIMAP/offlineimap/commit/7bc54d241cce -Bug: https://github.com/OfflineIMAP/offlineimap/issues/160 -Bug-Debian: https://bugs.debian.org/809676 ---- - offlineimap/ui/Curses.py | 15 ++++++++------- - 1 file changed, 8 insertions(+), 7 deletions(-) - -diff --git a/offlineimap/ui/Curses.py b/offlineimap/ui/Curses.py -index 152c5d6..e97bed0 100644 ---- a/offlineimap/ui/Curses.py -+++ b/offlineimap/ui/Curses.py -@@ -315,11 +315,11 @@ class CursesLogHandler(logging.StreamHandler): - y,x = self.ui.logwin.getyx() - if y or x: self.ui.logwin.addch(10) # no \n before 1st item - self.ui.logwin.addstr(log_str, color) -+ self.ui.logwin.noutrefresh() -+ self.ui.stdscr.refresh() - finally: - self.ui.unlock() - self.ui.tframe_lock.release() -- self.ui.logwin.noutrefresh() -- self.ui.stdscr.refresh() - - class Blinkenlights(UIBase, CursesUtil): - """Curses-cased fancy UI. -@@ -611,11 +611,12 @@ class Blinkenlights(UIBase, CursesUtil): - color = curses.A_REVERSE - self.bannerwin.clear() # Delete old content (eg before resizes) - self.bannerwin.bkgd(' ', color) # Fill background with that color -- string = "%s %s"% (offlineimap.__productname__, -- offlineimap.__version__) -- self.bannerwin.addstr(0, 0, string, color) -- self.bannerwin.addstr(0, self.width -len(offlineimap.__copyright__) -1, -- offlineimap.__copyright__, color) -+ string = "%s %s" % (offlineimap.__productname__, -+ offlineimap.__version__) -+ spaces = " " * max(1, (self.width - len(offlineimap.__copyright__) -+ - len(string) - 1)) -+ string = "%s%s%s" % (string, spaces, offlineimap.__copyright__) -+ self.bannerwin.addnstr(0, 0, string, self.width - 1, color) - self.bannerwin.noutrefresh() - - def draw_logwin(self): diff -Nru offlineimap-7.0.12+dfsg1/debian/patches/Document-the-configuration-options-in-the-manpage.patch offlineimap-7.1.2+dfsg1/debian/patches/Document-the-configuration-options-in-the-manpage.patch --- offlineimap-7.0.12+dfsg1/debian/patches/Document-the-configuration-options-in-the-manpage.patch 2017-05-03 15:29:54.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/patches/Document-the-configuration-options-in-the-manpage.patch 2017-07-11 15:00:33.000000000 +0000 @@ -11,7 +11,7 @@ 1 file changed, 7 insertions(+) diff --git a/docs/offlineimap.txt b/docs/offlineimap.txt -index fc4a966..964d499 100644 +index b5f4229..bc4dc39 100644 --- a/docs/offlineimap.txt +++ b/docs/offlineimap.txt @@ -195,6 +195,13 @@ It is possible to manually remove intermediate files in '/mbnames/'. diff -Nru offlineimap-7.0.12+dfsg1/debian/patches/Do-not-use-the-Internet-to-fetch-DTD.patch offlineimap-7.1.2+dfsg1/debian/patches/Do-not-use-the-Internet-to-fetch-DTD.patch --- offlineimap-7.0.12+dfsg1/debian/patches/Do-not-use-the-Internet-to-fetch-DTD.patch 2017-05-03 15:29:54.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/patches/Do-not-use-the-Internet-to-fetch-DTD.patch 2017-07-11 15:00:33.000000000 +0000 @@ -11,10 +11,10 @@ 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/Makefile b/docs/Makefile -index 0753cfe..185c46c 100644 +index 60148da..1476e19 100644 --- a/docs/Makefile +++ b/docs/Makefile -@@ -19,10 +19,10 @@ $(HTML_TARGETS): %.html : %.rst +@@ -28,10 +28,10 @@ offlineimapui.html: offlineimapui.txt man: offlineimap.1 offlineimapui.7 offlineimap.1: offlineimap.txt offlineimap.known_issues.txt diff -Nru offlineimap-7.0.12+dfsg1/debian/patches/series offlineimap-7.1.2+dfsg1/debian/patches/series --- offlineimap-7.0.12+dfsg1/debian/patches/series 2017-05-03 15:29:54.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/patches/series 2017-07-11 15:00:33.000000000 +0000 @@ -1,3 +1,2 @@ Do-not-use-the-Internet-to-fetch-DTD.patch Document-the-configuration-options-in-the-manpage.patch -Acquire-lock-before-updating-the-CursesLogHandler-window.patch diff -Nru offlineimap-7.0.12+dfsg1/debian/rules offlineimap-7.1.2+dfsg1/debian/rules --- offlineimap-7.0.12+dfsg1/debian/rules 2017-03-24 16:46:38.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/rules 2017-07-11 15:00:33.000000000 +0000 @@ -22,7 +22,6 @@ # Reproducable builds # https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal export PATH := $(CURDIR)/debian/strip-nondeterminism:$(PATH) -export SOURCE_DATE_EPOCH = $(shell date -d "$$(dpkg-parsechangelog -SDate)" +%s) %: diff -Nru offlineimap-7.0.12+dfsg1/debian/strip-nondeterminism/a2x offlineimap-7.1.2+dfsg1/debian/strip-nondeterminism/a2x --- offlineimap-7.0.12+dfsg1/debian/strip-nondeterminism/a2x 2017-03-24 16:46:38.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/debian/strip-nondeterminism/a2x 2017-07-11 15:00:33.000000000 +0000 @@ -3,10 +3,4 @@ # Eventually the upstream tool should support SOURCE_DATE_EPOCH internally. test -n "$SOURCE_DATE_EPOCH" || { echo >&2 "$0: SOURCE_DATE_EPOCH not set"; exit 255; } - -if faketime 'last friday' date > /dev/null 2> /dev/null; then - exec faketime -f "`TZ=UTC date -d @$SOURCE_DATE_EPOCH +'%Y-%m-%d %H:%M:%S'`" \ - /usr/bin/a2x "$@" -else - exec /usr/bin/a2x "$@" -fi +exec env NO_FAKE_STAT=1 faketime -f "$(TZ=UTC date -d "@$SOURCE_DATE_EPOCH" +'%Y-%m-%d %H:%M:%S')" /usr/bin/a2x "$@" diff -Nru offlineimap-7.0.12+dfsg1/docs/Makefile offlineimap-7.1.2+dfsg1/docs/Makefile --- offlineimap-7.0.12+dfsg1/docs/Makefile 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/docs/Makefile 2017-07-10 14:58:50.000000000 +0000 @@ -16,6 +16,15 @@ $(HTML_TARGETS): %.html : %.rst $(RST2HTML) $? $@ +manhtml: offlineimap.html offlineimapui.html + +offlineimap.html: offlineimap.txt offlineimap.known_issues.txt + a2x -v -d manpage -D manhtml -f xhtml $< + +offlineimapui.html: offlineimapui.txt + a2x -v -d manpage -D manhtml -f xhtml $< + + man: offlineimap.1 offlineimapui.7 offlineimap.1: offlineimap.txt offlineimap.known_issues.txt @@ -30,13 +39,15 @@ websitedoc: ./website-doc.sh releases ./website-doc.sh api + ./website-doc.sh html ./website-doc.sh contrib clean: $(RM) -f $(HTML_TARGETS) $(RM) -f offlineimap.1 $(RM) -f offlineimap.7 + $(RM) -f manhtml/* $(RM) -rf html/* - -find ./docs -name '*.html' -exec rm -f {} \; + -find . -name '*.html' -exec rm -f {} \; .PHONY: clean doc diff -Nru offlineimap-7.0.12+dfsg1/docs/offlineimap.known_issues.txt offlineimap-7.1.2+dfsg1/docs/offlineimap.known_issues.txt --- offlineimap-7.0.12+dfsg1/docs/offlineimap.known_issues.txt 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/docs/offlineimap.known_issues.txt 2017-07-10 14:58:50.000000000 +0000 @@ -1,4 +1,15 @@ +* Deletions. ++ +While in usual run the deletions are propagated. To prevent from data loss, +removing a folder makes offlineimap re-sync the folder. However, propagating the +removal of the whole content of a folder can happen in the two following cases: + + - The whole content of a folder is deleted but the folder directory still + exists. + + - The parent directory of the folder was deleted. + * SSL3 write pending. + Users enabling SSL may hit a bug about "SSL3 write pending". If so, the @@ -91,6 +102,11 @@ when offlineimap first syncs a pre-existing Maildir. In the latter case, it could appear as if a noticeable and random subset of old messages are synced. +* Offlineimap hangs. ++ +When having unexpected hangs it's advised to set `singlethreadperfolder' to +'yes', especially when in IMAP/IMAP mode (no maildir). + * Passwords in netrc. + Offlineimap doesn't know how to retrieve passwords when more than one account is @@ -109,7 +125,7 @@ - oauth2handler got: {u'error': u'invalid_grant'} + -In such case, we had report that generating a new refesh token from the same +In such case, we had reports that generating a new refresh token from the same client ID and secret can help. + .Google documentation on "invalid_grant" @@ -140,3 +156,12 @@ and to know more. +* "does not have message with UID" with Microsoft servers ++ + `ERROR: IMAP server 'Server ### Remote' does not have a message with UID 'xxx'` ++ +Microsoft IMAP servers are not compliant with the RFC. It is currently required +to folderfilter some faulting folders. See +http://www.offlineimap.org/doc/FAQ.html#exchange-and-office365 for a detailed +list. + diff -Nru offlineimap-7.0.12+dfsg1/docs/offlineimap.txt offlineimap-7.1.2+dfsg1/docs/offlineimap.txt --- offlineimap-7.0.12+dfsg1/docs/offlineimap.txt 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/docs/offlineimap.txt 2017-07-10 14:58:50.000000000 +0000 @@ -225,17 +225,17 @@ OfflineImap caches the state of the synchronisation to e.g. be able to determine if a mail has been added or deleted on either side. + -The historical status cache is a plain text file that writes out the complete -file for each single new message (or even changed flag) to a temporary file. If -you have plenty of files in a folder, this is a few hundred kilo to megabytes -for each mail and is bound to make things slow. The latest default status cache -is sqlite. This saves plenty of disk activity. The sqlite engine and the Python -sqlite module must be installed. Enable the 'status_backend = plain' setting in -'offlineimap.conf' for legacy compatibility with versions prior to '6.4.0'. -+ -If you switch the backend from plain to sqlite, you may want to delete the old -cache directory in '/Account-/LocalStatus' manually (the -sqlite cache stands in the 'LocalStatus-sqlite' folder). +The historical status cache was a plain text file that was writing out the +complete file for each single new message (or even changed flag) to a temporary +file. If there was plenty of files in a folder this was bound to make things +slow. The latest status cache is sqlite. This saves plenty of disk activity. ++ +The historical plain status cache is not supported anymore but migrating from a +very old installation using the plain text cache is still supported. In this +case, you may want to delete the old cache directory in +'/Account-/LocalStatus' manually (the sqlite cache stands in +the 'LocalStatus-sqlite' folder). First, make sure you have run the new version +of offlineimap for all your accounts so that the status cache was migrated. 4. Use quick sync. + @@ -315,10 +315,10 @@ accounts that are configured to 'autorefresh'. In effect, this will trigger a full sync of all accounts to be performed as soon as possible. -* If sent a SIGUSR2, it will stop 'autorefresh' mode for all accounts. That -is, accounts will abort any current sleep and will exit after a currently -running synchronization has finished. This signal can be used to gracefully -exit out of a running offlineimap "daemon". +* If sent a SIGUSR2 or SIGABRT, it will stop 'autorefresh' mode for all +accounts. That is, accounts will abort any current sleep and will exit after a +currently running synchronization has finished. This signal can be used to +gracefully exit out of a running offlineimap "daemon". * SIGTERM, SIGINT, SIGHUP are all treated to gracefully terminate as soon as possible. This means it will finish syncing the current folder in each diff -Nru offlineimap-7.0.12+dfsg1/docs/website-doc.sh offlineimap-7.1.2+dfsg1/docs/website-doc.sh --- offlineimap-7.0.12+dfsg1/docs/website-doc.sh 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/docs/website-doc.sh 2017-07-10 14:58:50.000000000 +0000 @@ -55,10 +55,12 @@ echo "Building Jekyll data: $VERSIONS_YML" # Erase previous content. echo "$HEADER" > "$VERSIONS_YML" + echo "# However, it's correct to /remove/ old API docs here." + echo "# While at it, don't forget to adjust the _doc/versions directory." for version in $(ls "$DESTBASE" -1 | sort -nr) do echo "- $version" - done >> "$VERSIONS_YML" + done | sort -V >> "$VERSIONS_YML" } @@ -124,6 +126,15 @@ rm -f "$ANNOUNCES_YML_TMP" } +function manhtml () { + set -e + + cd ./docs + make manhtml + cd .. + cp -afv ./docs/manhtml/* "$DOCBASE" +} + exit_code=0 test "n$ARGS" = 'n' && ARGS='usage' # no option passed @@ -137,6 +148,9 @@ "napi") api ;; + "nhtml") + manhtml + ;; "ncontrib") contrib ;; diff -Nru offlineimap-7.0.12+dfsg1/.github/CODEOWNERS offlineimap-7.1.2+dfsg1/.github/CODEOWNERS --- offlineimap-7.0.12+dfsg1/.github/CODEOWNERS 1970-01-01 00:00:00.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/.github/CODEOWNERS 2017-07-10 14:58:50.000000000 +0000 @@ -0,0 +1,19 @@ +# This is a comment. +# Each line is a file pattern followed by one or more owners. + +# These owners will be the default owners for everything in the repo. +# Unless a later match takes precedence, @global-owner1 and @global-owner2 +# will be requested for review when someone opens a pull request. +#* @global-owner1 @global-owner2 + +# Order is important; the last matching pattern takes the most precedence. +# When someone opens a pull request that only modifies JS files, only @js-owner +# and not the global owner(s) will be requested for a review. +#*.js @js-owner + +# You can also use email addresses if you prefer. They'll be used to look up +# users just like we do for commit author emails. +#docs/* docs@example.com + + +* @chris001 diff -Nru offlineimap-7.0.12+dfsg1/.gitignore offlineimap-7.1.2+dfsg1/.gitignore --- offlineimap-7.0.12+dfsg1/.gitignore 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/.gitignore 2017-07-10 14:58:50.000000000 +0000 @@ -1,16 +1,18 @@ -# Backups. +# Editors/IDEs .*.swp .*.swo -*.html *~ +tags + # websites. /website/ /wiki/ + # Generated files. /docs/dev-doc/ /build/ +*.html +*.css *.pyc offlineimap.1 offlineimapui.7 -# Editors/IDEs -tags diff -Nru offlineimap-7.0.12+dfsg1/MAINTAINERS.rst offlineimap-7.1.2+dfsg1/MAINTAINERS.rst --- offlineimap-7.0.12+dfsg1/MAINTAINERS.rst 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/MAINTAINERS.rst 2017-07-10 14:58:50.000000000 +0000 @@ -4,39 +4,66 @@ ======== - Abdó Roig-Maranges - - email: abdo.roig at gmail.com - - github: aroig + - email: abdo.roig at gmail.com + - github: aroig - Ben Boeckel - - email: mathstuf at gmail.com - - github: mathstuf + - email: mathstuf at gmail.com + - github: mathstuf + +- benutzer193 + - email: registerbn at gmail.com + - github: benutzer193 + +- Chris Coleman + - email: christocoleman at yahoo.com + - github: chris001 + +- Darshit Shah + - email: darnir at gmail.com + - github: darnir - Eygene Ryabinkin - - email: rea at freebsd.org - - github: konvpalto - - other: FreeBSD maintainer + - email: rea at freebsd.org + - github: konvpalto + - other: FreeBSD maintainer - Igor Almeida - - email: igor.contato at gmail.com - - github: igoralmeida + - email: igor.contato at gmail.com + - github: igoralmeida - Ilias Tsitsimpis - - email: i.tsitsimpis at gmail.com - - github: iliastsi - - other: Debian maintainer + - email: i.tsitsimpis at gmail.com + - github: iliastsi + - other: Debian maintainer + +- "J" + - email: offlineimap at 927589452.de + - github: 927589452 + - other: FreeBSD user - Łukasz Żarnowiecki - - email: dolohow at outlook.com - - github: dolohow + - email: dolohow at outlook.com + - github: dolohow - Nicolas Sebrecht - - email: nicolas.s-dev at laposte.net - - github: nicolas33 + - email: nicolas.s-dev at laposte.net + - github: nicolas33 + - system: Linux + +- Rainer M Krug + - email: Rainer at krugs.de + - github: rkrug + - system: OSX + +- Remi Locherer + - email: remi.locherer at relo.ch + - system: OpenBSD maintainer - Sebastian Spaeth - - email: sebastian at sspaeth.de - - github: spaetz - - other: left the project but still responding + - email: sebastian at sspaeth.de + - github: spaetz + - other: left the project but still responding Testers @@ -44,11 +71,16 @@ - Abdó Roig-Maranges - Ben Boeckel +- Chris Coleman +- Darshit Shah - Eygene Ryabinkin - Igor Almeida - Ilias Tsitsimpis +- "J" - Łukasz Żarnowiecki - Nicolas Sebrecht +- Rainer M Krug +- Remi Locherer Maintainers diff -Nru offlineimap-7.0.12+dfsg1/Makefile offlineimap-7.1.2+dfsg1/Makefile --- offlineimap-7.0.12+dfsg1/Makefile 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/Makefile 2017-07-10 14:58:50.000000000 +0000 @@ -38,7 +38,7 @@ -find . -name '.cache*' -exec rm -f {} \; -rm -f manpage.links manpage.refs 2>/dev/null -find . -name auth -exec rm -vf {}/password {}/username \; - @$(MAKE) -C clean + -$(MAKE) -C docs clean .PHONY: docs docs: diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/accounts.py offlineimap-7.1.2+dfsg1/offlineimap/accounts.py --- offlineimap-7.0.12+dfsg1/offlineimap/accounts.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/accounts.py 2017-07-10 14:58:50.000000000 +0000 @@ -15,7 +15,7 @@ # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA from subprocess import Popen, PIPE -from threading import Event +from threading import Event, Lock import os import time from sys import exc_info @@ -29,6 +29,9 @@ from offlineimap.threadutil import InstanceLimitedThread FOLDER_NAMESPACE = 'LIMITED_FOLDER_' +# Key: account name, Value: Dict of Key: remotefolder name, Value: lock. +SYNC_MUTEXES = {} +SYNC_MUTEXES_LOCK = Lock() try: import fcntl @@ -306,6 +309,9 @@ remotefolder.getvisiblename(). replace(self.remoterepos.getsep(), self.localrepos.getsep())) + + # The syncrunner will loop on this method. This means it is called more than + # once during the run. def __sync(self): """Synchronize the account once, then return. @@ -430,11 +436,35 @@ self.ui.error(e, exc_info()[2], msg="Calling hook") +#XXX: This function should likely be refactored. This should not be passed the +# account instance. def syncfolder(account, remotefolder, quick): """Synchronizes given remote folder for the specified account. - Filtered folders on the remote side will not invoke this function. However, - this might be called in a concurrently.""" + Filtered folders on the remote side will not invoke this function. + + When called in concurrently for the same localfolder, syncs are + serialized.""" + + def acquire_mutex(): + account_name = account.getname() + localfolder_name = localfolder.getfullname() + + with SYNC_MUTEXES_LOCK: + if SYNC_MUTEXES.get(account_name) is None: + SYNC_MUTEXES[account_name] = {} + # The localfolder full name is good to uniquely identify the sync + # transaction. + if SYNC_MUTEXES[account_name].get(localfolder_name) is None: + #XXX: This lock could be an external file lock so we can remove + # the lock at the account level. + SYNC_MUTEXES[account_name][localfolder_name] = Lock() + + # Acquire the lock. + SYNC_MUTEXES[account_name][localfolder_name].acquire() + + def release_mutex(): + SYNC_MUTEXES[account.getname()][localfolder.getfullname()].release() def check_uid_validity(): # If either the local or the status folder has messages and @@ -459,23 +489,21 @@ def cachemessagelists_upto_date(date): """Returns messages with uid > min(uids of messages newer than date).""" - # Warning: this makes sense only if the cached list is empty. - localfolder.cachemessagelist(min_date=date) - check_uid_validity() - # Local messagelist had date restriction applied already. Restrict - # sync to messages with UIDs >= min_uid from this list. - # - # Local messagelist might contain new messages (with uid's < 0). - positive_uids = [uid for uid in localfolder.getmessageuidlist() if uid > 0] - if len(positive_uids) > 0: - remotefolder.cachemessagelist(min_uid=min(positive_uids)) + remotefolder.cachemessagelist( + min_date=time.gmtime(time.mktime(date) + 24*60*60)) + uids = remotefolder.getmessageuidlist() + localfolder.dropmessagelistcache() + if len(uids) > 0: + localfolder.cachemessagelist(min_uid=min(uids)) else: - # No messages with UID > 0 in range in localfolder. - # date restriction was applied with respect to local dates but - # remote folder timezone might be different from local, so be - # safe and make sure the range isn't bigger than in local. - remotefolder.cachemessagelist( - min_date=time.gmtime(time.mktime(date) + 24*60*60)) + # Remote folder UIDs list is empty for the given range. We still + # might have valid local UIDs for this range (e.g.: new local + # emails). + localfolder.cachemessagelist(min_date=date) + uids = localfolder.getmessageuidlist() + if len(uids) > 0: + # Update the remote cache list for this new min(uids). + remotefolder.cachemessagelist(min_uid=min(uids)) def cachemessagelists_startdate(new, partial, date): """Retrieve messagelists when startdate has been set for @@ -528,6 +556,9 @@ # Load local folder. localfolder = account.get_local_folder(remotefolder) + # Acquire the mutex to start syncing. + acquire_mutex() + # Add the folder to the mbnames mailboxes. mbnames.add(account.name, localrepos.getlocalroot(), localfolder.getname()) @@ -616,3 +647,5 @@ if folder in locals(): locals()[folder].dropmessagelistcache() statusfolder.closefiles() + # Release the mutex of this sync transaction. + release_mutex() diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/folder/Gmail.py offlineimap-7.1.2+dfsg1/offlineimap/folder/Gmail.py --- offlineimap-7.0.12+dfsg1/offlineimap/folder/Gmail.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/folder/Gmail.py 2017-07-10 14:58:50.000000000 +0000 @@ -71,7 +71,7 @@ (probably severity MESSAGE) if e.g. no message with this UID could be found. """ - data = self._fetch_from_imap(str(uid), 2) + data = self._fetch_from_imap(str(uid), self.retrycount) # data looks now e.g. #[('320 (X-GM-LABELS (...) UID 17061 BODY[] {2565}','msgbody....')] diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/folder/IMAP.py offlineimap-7.1.2+dfsg1/offlineimap/folder/IMAP.py --- offlineimap-7.0.12+dfsg1/offlineimap/folder/IMAP.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/folder/IMAP.py 2017-07-10 14:58:50.000000000 +0000 @@ -55,6 +55,9 @@ # self.ui is set in BaseFolder. self.imap_query = ['BODY.PEEK[]'] + # number of times to retry fetching messages + self.retrycount = self.repository.getconfint('retrycount', 2) + fh_conf = self.repository.account.getconf('filterheaders', '') self.filterheaders = [h for h in re.split(r'\s*,\s*', fh_conf) if h] @@ -305,7 +308,7 @@ this UID could be found. """ - data = self._fetch_from_imap(str(uid), 2) + data = self._fetch_from_imap(str(uid), self.retrycount) # Data looks now e.g. # [('320 (UID 17061 BODY[] {2565}','msgbody....')] @@ -389,9 +392,12 @@ self.ui.debug('imap', '__savemessage_searchforheader: matchinguids now ' + repr(matchinguids)) if len(matchinguids) != 1 or matchinguids[0] is None: - raise ValueError("While attempting to find UID for message with " - "header %s, got wrong-sized matchinguids of %s"% - (headername, str(matchinguids))) + raise OfflineImapError( + "While attempting to find UID for message with " + "header %s, got wrong-sized matchinguids of %s"% + (headername, str(matchinguids)), + OfflineImapError.ERROR.MESSAGE + ) return int(matchinguids[0]) def __savemessage_fetchheaders(self, imapobj, headername, headervalue): @@ -442,24 +448,46 @@ raise OfflineImapError('Error fetching mail headers: %s'% '. '.join(result[1]), OfflineImapError.ERROR.MESSAGE) + # result is like: + # [ + # ('185 (RFC822.HEADER {1789}', '... mail headers ...'), ' UID 2444)', + # ('186 (RFC822.HEADER {1789}', '... 2nd mail headers ...'), ' UID 2445)' + # ] result = result[1] - found = 0 + found = None + # item is like: + # ('185 (RFC822.HEADER {1789}', '... mail headers ...'), ' UID 2444)' for item in result: - if found == 0 and type(item) == type( () ): + if found is None and type(item) == tuple: # Walk just tuples. if re.search("(?:^|\\r|\\n)%s:\s*%s(?:\\r|\\n)"% (headername, headervalue), item[1], flags=re.IGNORECASE): - found = 1 - elif found == 1: - if type(item) == type (""): + found = item[0] + elif found is not None: + if type(item) == type(""): uid = re.search("UID\s+(\d+)", item, flags=re.IGNORECASE) if uid: return int(uid.group(1)) else: - self.ui.warn("Can't parse FETCH response, can't find UID: %s", result.__repr__()) + # This parsing is for Davmail. + # https://github.com/OfflineIMAP/offlineimap/issues/479 + # item is like: + # ')' + # and item[0] stored in "found" is like: + # '1694 (UID 1694 RFC822.HEADER {1294}' + uid = re.search("\d+\s+\(UID\s+(\d+)", found, flags=re.IGNORECASE) + if uid: + return int(uid.group(1)) + + self.ui.warn("Can't parse FETCH response, can't find UID in %s"% + item + ) + self.ui.debug('imap', "Got: %s"% repr(result)) else: - self.ui.warn("Can't parse FETCH response, we awaited string: %s", result.__repr__()) + self.ui.warn("Can't parse FETCH response, we awaited string: %s"% + repr(item) + ) return 0 @@ -676,9 +704,15 @@ resp = imapobj._get_untagged_response('APPENDUID') if resp == [None] or resp is None: self.ui.warn("Server supports UIDPLUS but got no APPENDUID " - "appending a message.") + "appending a message. Got: %s."% str(resp)) return 0 - uid = int(resp[-1].split(' ')[1]) + try: + uid = int(resp[-1].split(' ')[1]) + except ValueError as e: + uid = 0 # Definetly not what we should have. + except Exception as e: + raise OfflineImapError("Unexpected response: %s"% str(resp), + OfflineImapError.ERROR.MESSAGE) if uid == 0: self.ui.warn("savemessage: Server supports UIDPLUS, but" " we got no usable uid back. APPENDUID reponse was " @@ -694,13 +728,14 @@ 'UID failed. Search headers manually.') uid = self.__savemessage_fetchheaders(imapobj, headername, headervalue) - self.ui.warn('imap', "savemessage: Searching mails for new " - "Message-ID failed. Could not determine new UID.") + self.ui.warn("savemessage: Searching mails for new " + "Message-ID failed. Could not determine new UID " + "on %s."% self.getname()) finally: if imapobj: self.imapserver.releaseconnection(imapobj) - if uid: # Avoid UID FETCH 0 crash happening later on + if uid: # Avoid UID FETCH 0 crash happening later on. self.messagelist[uid] = self.msglist_item_initializer(uid) self.messagelist[uid]['flags'] = flags @@ -757,13 +792,15 @@ # exactly one response. if res_type == 'OK': data = [res for res in data if not isinstance(res, str)] - # Could not fetch message. + + # Could not fetch message. Note: it is allowed by rfc3501 to return any + # data for the UID FETCH command. if data == [None] or res_type != 'OK' or len(data) != 1: severity = OfflineImapError.ERROR.MESSAGE reason = "IMAP server '%s' failed to fetch messages UID '%s'."\ - "Server responded: %s %s"% (self.getrepository(), uids, + " Server responded: %s %s"% (self.getrepository(), uids, res_type, data) - if data == [None]: + if data == [None] or len(data) < 1: # IMAP server did not find a message with this UID. reason = "IMAP server '%s' does not have a message "\ "with UID '%s'"% (self.getrepository(), uids) diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/folder/LocalStatusSQLite.py offlineimap-7.1.2+dfsg1/offlineimap/folder/LocalStatusSQLite.py --- offlineimap-7.0.12+dfsg1/offlineimap/folder/LocalStatusSQLite.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/folder/LocalStatusSQLite.py 2017-07-10 14:58:50.000000000 +0000 @@ -1,5 +1,5 @@ # Local status cache virtual folder: SQLite backend -# Copyright (C) 2009-2016 Stewart Smith and contributors. +# Copyright (C) 2009-2017 Stewart Smith and contributors. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -282,16 +282,17 @@ def saveall(self): """Saves the entire messagelist to the database.""" - data = [] - for uid, msg in self.messagelist.items(): - mtime = msg['mtime'] - flags = ''.join(sorted(msg['flags'])) - labels = ', '.join(sorted(msg['labels'])) - data.append((uid, flags, mtime, labels)) - - self.__sql_write('INSERT OR REPLACE INTO status ' - '(id,flags,mtime,labels) VALUES (?,?,?,?)', - data, executemany=True) + with self._databaseFileLock.getLock(): + data = [] + for uid, msg in self.messagelist.items(): + mtime = msg['mtime'] + flags = ''.join(sorted(msg['flags'])) + labels = ', '.join(sorted(msg['labels'])) + data.append((uid, flags, mtime, labels)) + + self.__sql_write('INSERT OR REPLACE INTO status ' + '(id,flags,mtime,labels) VALUES (?,?,?,?)', + data, executemany=True) # Following some pure SQLite functions, where we chose to use diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/folder/UIDMaps.py offlineimap-7.1.2+dfsg1/offlineimap/folder/UIDMaps.py --- offlineimap-7.0.12+dfsg1/offlineimap/folder/UIDMaps.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/folder/UIDMaps.py 2017-07-10 14:58:50.000000000 +0000 @@ -142,8 +142,22 @@ for luid in self.diskl2r.keys(): if not luid in reallist: ruid = self.diskl2r[luid] - del self.diskr2l[ruid] - del self.diskl2r[luid] + #XXX: the following KeyError are sightly unexpected. This + # would require more digging to understand how it's + # possible. + errorMessage = ("unexpected error: key {} was not found " + "in memory, see " + "https://github.com/OfflineIMAP/offlineimap/issues/445" + " to know more." + ) + try: + del self.diskr2l[ruid] + except KeyError as e: + self.ui.warn(errorMessage.format(ruid)) + try: + del self.diskl2r[luid] + except KeyError as e: + self.ui.warn(errorMessage.format(ruid)) # Now, assign negative UIDs to local items. self._savemaps() @@ -253,8 +267,12 @@ newluid = self._mb.savemessage(-1, content, flags, rtime) if newluid < 1: - raise ValueError("Backend could not find uid for message, " - "returned %s"% newluid) + raise OfflineImapError("server of repository '%s' did not return " + "a valid UID (got '%s') for UID '%s' from '%s'"% ( + self._mb.getname(), newluid, uid, self.getname() + ), + OfflineImapError.ERROR.MESSAGE + ) with self.maplock: self.diskl2r[newluid] = uid self.diskr2l[uid] = newluid diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/imapserver.py offlineimap-7.1.2+dfsg1/offlineimap/imapserver.py --- offlineimap-7.0.12+dfsg1/offlineimap/imapserver.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/imapserver.py 2017-07-10 14:58:50.000000000 +0000 @@ -134,27 +134,43 @@ # In order to support proxy connection, we have to override the # default socket instance with our own socksified socket instance. # We add this option to bypass the GFW in China. + self.proxied_socket = self._get_proxy('proxy', socket.socket) + + # Turns out that the GFW in China is no longer blocking imap.gmail.com + # However accounts.google.com (for oauth2) definitey is. Therefore + # it is not strictly necessary to use a proxy for *both* IMAP *and* + # oauth2, so a new option is added: authproxy. + + # Set proxy for use in authentication (only) if desired. + # If not set, is same as proxy option (compatible with current configs) + # To use a proxied_socket but not an authproxied_socket + # set authproxy = '' in config + self.authproxied_socket = self._get_proxy('authproxy', + self.proxied_socket) + + def _get_proxy(self, proxysection, dfltsocket): _account_section = 'Account ' + self.repos.account.name - if not self.config.has_option(_account_section, 'proxy'): - self.proxied_socket = socket.socket - else: - proxy = self.config.get(_account_section, 'proxy') - # Powered by PySocks. - try: - import socks - proxy_type, host, port = proxy.split(":") - port = int(port) - socks.setdefaultproxy(getattr(socks, proxy_type), host, port) - self.proxied_socket = socks.socksocket - except ImportError: - self.ui.warn("PySocks not installed, ignoring proxy option.") - self.proxied_socket = socket.socket - except (AttributeError, ValueError) as e: - self.ui.warn("Bad proxy option %s for account %s: %s " - "Ignoring proxy option."% - (proxy, self.repos.account.name, e)) - self.proxied_socket = socket.socket + if not self.config.has_option(_account_section, proxysection): + return dfltsocket + proxy = self.config.get(_account_section, proxysection) + if proxy == '': + # explicitly set no proxy (overrides default return of dfltsocket) + return socket.socket + # Powered by PySocks. + try: + import socks + proxy_type, host, port = proxy.split(":") + port = int(port) + socks.setdefaultproxy(getattr(socks, proxy_type), host, port) + return socks.socksocket + except ImportError: + self.ui.warn("PySocks not installed, ignoring proxy option.") + except (AttributeError, ValueError) as e: + self.ui.warn("Bad proxy option %s for account %s: %s " + "Ignoring %s option."% + (proxy, self.repos.account.name, e, proxysection)) + return dfltsocket def __getpassword(self): """Returns the server password or None""" @@ -205,10 +221,6 @@ return retval def __xoauth2handler(self, response): - if self.oauth2_refresh_token is None \ - and self.oauth2_access_token is None: - return None - if self.oauth2_access_token is None: if self.oauth2_request_url is None: raise OfflineImapError("No remote oauth2_request_url for " @@ -227,7 +239,7 @@ self.ui.debug('imap', 'xoauth2handler: params "%s"'% params) original_socket = socket.socket - socket.socket = self.proxied_socket + socket.socket = self.authproxied_socket try: response = urllib.urlopen( self.oauth2_request_url, urllib.urlencode(params)).read() @@ -337,6 +349,10 @@ return True def __authn_xoauth2(self, imapobj): + if self.oauth2_refresh_token is None \ + and self.oauth2_access_token is None: + return False + imapobj.authenticate('XOAUTH2', self.__xoauth2handler) return True @@ -511,6 +527,8 @@ elif self.usessl: self.ui.connecting( self.repos.getname(), self.hostname, self.port) + self.ui.debug('imap', "%s: level '%s', version '%s'"% + (self.repos.getname(), self.tlslevel, self.sslversion)) imapobj = imaplibutil.WrappedIMAP4_SSL( host=self.hostname, port=self.port, @@ -601,7 +619,8 @@ if self.port != 993: reason = "Could not connect via SSL to host '%s' and non-s"\ "tandard ssl port %d configured. Make sure you connect"\ - " to the correct port."% (self.hostname, self.port) + " to the correct port. Got: %s"% ( + self.hostname, self.port, e) else: reason = "Unknown SSL protocol connecting to host '%s' for "\ "repository '%s'. OpenSSL responded:\n%s"\ diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/__init__.py offlineimap-7.1.2+dfsg1/offlineimap/__init__.py --- offlineimap-7.0.12+dfsg1/offlineimap/__init__.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/__init__.py 2017-07-10 14:58:50.000000000 +0000 @@ -2,8 +2,8 @@ __productname__ = 'OfflineIMAP' # Expecting trailing "-rcN" or "" for stable releases. -__version__ = "7.0.12" -__copyright__ = "Copyright 2002-2016 John Goerzen & contributors" +__version__ = "7.1.2" +__copyright__ = "Copyright 2002-2017 John Goerzen & contributors" __author__ = "John Goerzen" __author_email__= "offlineimap-project@lists.alioth.debian.org" __description__ = "Disconnected Universal IMAP Mail Synchronization/Reader Support" diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/init.py offlineimap-7.1.2+dfsg1/offlineimap/init.py --- offlineimap-7.0.12+dfsg1/offlineimap/init.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/init.py 2017-07-10 14:58:50.000000000 +0000 @@ -1,5 +1,5 @@ # OfflineIMAP initialization code -# Copyright (C) 2002-2016 John Goerzen & contributors +# Copyright (C) 2002-2017 John Goerzen & contributors # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -72,6 +72,17 @@ oi.run() """ + def get_env_info(self): + info = "imaplib2 v%s (%s), Python v%s"% ( + imaplib.__version__, imaplib.DESC, PYTHON_VERSION + ) + try: + import ssl + info = "%s, %s"% (info, ssl.OPENSSL_VERSION) + except: + pass + return info + def run(self): """Parse the commandline and invoke everything""" # next line also sets self.config and self.ui @@ -180,9 +191,8 @@ glob.set_options(options) if options.version: - print("offlineimap v%s, imaplib2 v%s (%s), Python v%s"% ( - offlineimap.__version__, imaplib.__version__, imaplib.DESC, - PYTHON_VERSION) + print("offlineimap v%s, %s"% ( + offlineimap.__version__, self.get_env_info()) ) sys.exit(0) @@ -273,6 +283,7 @@ # Welcome blurb. self.ui.init_banner() + self.ui.info(self.get_env_info()) if options.debugtype: self.ui.logger.setLevel(logging.DEBUG) @@ -416,7 +427,7 @@ if sig == signal.SIGUSR1: # tell each account to stop sleeping accounts.Account.set_abort_event(self.config, 1) - elif sig == signal.SIGUSR2: + elif sig in (signal.SIGUSR2, signal.SIGABRT): # tell each account to stop looping getglobalui().warn("Terminating after this sync...") accounts.Account.set_abort_event(self.config, 2) @@ -442,6 +453,7 @@ signal.signal(signal.SIGHUP, sig_handler) signal.signal(signal.SIGUSR1, sig_handler) signal.signal(signal.SIGUSR2, sig_handler) + signal.signal(signal.SIGABRT, sig_handler) signal.signal(signal.SIGTERM, sig_handler) signal.signal(signal.SIGINT, sig_handler) signal.signal(signal.SIGQUIT, sig_handler) diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/mbnames.py offlineimap-7.1.2+dfsg1/offlineimap/mbnames.py --- offlineimap-7.0.12+dfsg1/offlineimap/mbnames.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/mbnames.py 2017-07-10 14:58:50.000000000 +0000 @@ -32,6 +32,7 @@ _mbnames = None +# Called at sync time for each folder. def add(accountname, folder_root, foldername): global _mbnames if _mbnames.is_enabled() is not True: @@ -41,12 +42,14 @@ _mbnames.addAccountFolder(accountname, folder_root, foldername) +# Called once. def init(conf, ui, dry_run): global _mbnames if _mbnames is None: _mbnames = _Mbnames(conf, ui, dry_run) +# Called once. def prune(accounts): global _mbnames if _mbnames.is_enabled() is True: @@ -55,6 +58,7 @@ _mbnames.pruneAll() +# Called once. def write(): """Write the mbnames file.""" @@ -66,6 +70,7 @@ _mbnames.write() +# Called as soon as all the folders are synced for the account. def writeIntermediateFile(accountname): """Write intermediate mbnames file.""" @@ -93,7 +98,8 @@ self._dryrun = dry_run def add(self, foldername): - self._foldernames.append(foldername) + if foldername not in self._foldernames: + self._foldernames.append(foldername) def get_folder_root(self): return self._folder_root diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/repository/IMAP.py offlineimap-7.1.2+dfsg1/offlineimap/repository/IMAP.py --- offlineimap-7.0.12+dfsg1/offlineimap/repository/IMAP.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/repository/IMAP.py 2017-07-10 14:58:50.000000000 +0000 @@ -222,7 +222,7 @@ return self.getconfint('remoteport', None) def getipv6(self): - return self.getconfboolean('ipv6', False) + return self.getconfboolean('ipv6', None) def getssl(self): return self.getconfboolean('ssl', True) @@ -454,17 +454,29 @@ listfunction = imapobj.list if self.getconfboolean('subscribedonly', False): listfunction = imapobj.lsub + try: - listresult = listfunction(directory=self.imapserver.reference)[1] + result, listresult = listfunction(directory=self.imapserver.reference) + if result != 'OK': + raise OfflineImapError("Could not list the folders for" + " repository %s. Server responded: %s"% + (self.name, self, str(listresult)), + OfflineImapError.ERROR.FOLDER) finally: self.imapserver.releaseconnection(imapobj) + for s in listresult: if s == None or \ (isinstance(s, str) and s == ''): # Bug in imaplib: empty strings in results from # literals. TODO: still relevant? continue - flags, delim, name = imaputil.imapsplit(s) + try: + flags, delim, name = imaputil.imapsplit(s) + except ValueError: + self.ui.error( + "could not correctly parse server response; got: %s"% s) + raise flaglist = [x.lower() for x in imaputil.flagsplit(flags)] if '\\noselect' in flaglist: continue diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/repository/LocalStatus.py offlineimap-7.1.2+dfsg1/offlineimap/repository/LocalStatus.py --- offlineimap-7.0.12+dfsg1/offlineimap/repository/LocalStatus.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/repository/LocalStatus.py 2017-07-10 14:58:50.000000000 +0000 @@ -1,5 +1,5 @@ # Local status cache repository support -# Copyright (C) 2002-2016 John Goerzen & contributors +# Copyright (C) 2002-2017 John Goerzen & contributors # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,30 +20,37 @@ from offlineimap.folder.LocalStatus import LocalStatusFolder from offlineimap.folder.LocalStatusSQLite import LocalStatusSQLiteFolder from offlineimap.repository.Base import BaseRepository +from offlineimap.error import OfflineImapError + class LocalStatusRepository(BaseRepository): def __init__(self, reposname, account): BaseRepository.__init__(self, reposname, account) - # class and root for all backends + # class and root for all backends. self.backends = {} self.backends['sqlite'] = { 'class': LocalStatusSQLiteFolder, 'root': os.path.join(account.getaccountmeta(), 'LocalStatus-sqlite') } - self.backends['plain'] = { 'class': LocalStatusFolder, 'root': os.path.join(account.getaccountmeta(), 'LocalStatus') } - # Set class and root for the configured backend - self.setup_backend(self.account.getconf('status_backend', 'sqlite')) + if self.account.getconf('status_backend', None) is not None: + raise OfflineImapError( + "the 'status_backend' configuration option is not supported" + " anymore; please, remove this configuration option.", + OfflineImapError.ERROR.REPO + ) + # Set class and root for sqlite. + self.setup_backend('sqlite') if not os.path.exists(self.root): os.mkdir(self.root, 0o700) - # self._folders is a dict of name:LocalStatusFolders() + # self._folders is a dict of name:LocalStatusFolders(). self._folders = {} def _instanciatefolder(self, foldername): @@ -55,10 +62,6 @@ self.root = self.backends[backend]['root'] self.LocalStatusFolderClass = self.backends[backend]['class'] - else: - raise SyntaxWarning("Unknown status_backend '%s' for account '%s'"% - (backend, self.account.name)) - def import_other_backend(self, folder): for bk, dic in self.backends.items(): # Skip folder's own type. diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/ui/Curses.py offlineimap-7.1.2+dfsg1/offlineimap/ui/Curses.py --- offlineimap-7.0.12+dfsg1/offlineimap/ui/Curses.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/ui/Curses.py 2017-07-10 14:58:50.000000000 +0000 @@ -315,11 +315,11 @@ y,x = self.ui.logwin.getyx() if y or x: self.ui.logwin.addch(10) # no \n before 1st item self.ui.logwin.addstr(log_str, color) + self.ui.logwin.noutrefresh() + self.ui.stdscr.refresh() finally: self.ui.unlock() self.ui.tframe_lock.release() - self.ui.logwin.noutrefresh() - self.ui.stdscr.refresh() class Blinkenlights(UIBase, CursesUtil): """Curses-cased fancy UI. @@ -611,11 +611,12 @@ color = curses.A_REVERSE self.bannerwin.clear() # Delete old content (eg before resizes) self.bannerwin.bkgd(' ', color) # Fill background with that color - string = "%s %s"% (offlineimap.__productname__, - offlineimap.__version__) - self.bannerwin.addstr(0, 0, string, color) - self.bannerwin.addstr(0, self.width -len(offlineimap.__copyright__) -1, - offlineimap.__copyright__, color) + string = "%s %s" % (offlineimap.__productname__, + offlineimap.__version__) + spaces = " " * max(1, (self.width - len(offlineimap.__copyright__) + - len(string) - 1)) + string = "%s%s%s" % (string, spaces, offlineimap.__copyright__) + self.bannerwin.addnstr(0, 0, string, self.width - 1, color) self.bannerwin.noutrefresh() def draw_logwin(self): diff -Nru offlineimap-7.0.12+dfsg1/offlineimap/virtual_imaplib2.py offlineimap-7.1.2+dfsg1/offlineimap/virtual_imaplib2.py --- offlineimap-7.0.12+dfsg1/offlineimap/virtual_imaplib2.py 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap/virtual_imaplib2.py 2017-07-10 14:58:50.000000000 +0000 @@ -25,7 +25,7 @@ DESC = None _SUPPORTED_RELEASE = 2 -_SUPPORTED_REVISION = 55 +_SUPPORTED_REVISION = 57 try: # Try any imaplib2 in PYTHONPATH first. This allows both maintainers of diff -Nru offlineimap-7.0.12+dfsg1/offlineimap.conf offlineimap-7.1.2+dfsg1/offlineimap.conf --- offlineimap-7.0.12+dfsg1/offlineimap.conf 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/offlineimap.conf 2017-07-10 14:58:50.000000000 +0000 @@ -228,6 +228,9 @@ # return foldername in allowed[accountname] # return False # +# For correct folderfilter with Microsoft servers, please see +# http://www.offlineimap.org/doc/FAQ.html#exchange-and-office365 +# #folderfilter = mbnames_folderfilter @@ -316,14 +319,6 @@ # This option stands in the [Account Test] section. # -# The historical backend is 'plain' which writes out the state in plain text -# files. See manual. -# -#status_backend = sqlite - - -# This option stands in the [Account Test] section. -# # If you have a limited amount of bandwidth available you can exclude larger # messages (e.g. those with large attachments etc). If you do this it will # appear to Offlineimap that these messages do not exist at all. They will not @@ -437,6 +432,21 @@ #proxy = SOCKS5:IP:9999 +# TESTING: This option stands in the [Account Test] section. +# +# Use authproxy connection for this account. Useful to bypass the GFW in China. +# Set this if you wish to use a proxy for authentication but not for IMAP. +# If not explicitly set, this option defaults to use the proxy socket +# (so as to be compatible with prior config files). +# If that is specifically NOT desired, use authproxy = '' +# +# To specify a proxy connection, join proxy type, host and port with colons. +# Available proxy types are SOCKS5, SOCKS4, HTTP. +# You also need to install PySocks through pip or your distro package manager. +# +#authproxy = SOCKS5:IP:9999 + + [Repository LocalExample] # Each repository requires a "type" declaration. The types supported for @@ -633,13 +643,14 @@ # This option stands in the [Repository RemoteExample] section. # # Whether or not to use STARTTLS. STARTTLS allows to upgrade a plain connection -# to TLS or SSL after negociation with the server. While a server might pretend +# to TLS or SSL after negotiation with the server. While a server might pretend # to support STARTTLS, the communication might not be properly established or # the secure tunnel might be broken in some way. In this case you might want to # disable STARTTLS. Unless you hit issues with STARTTLS, you are strongly # encouraged to keep STARTTLS enabled. # -# STARTTLS can be used even if the 'ssl' option is disabled. +# STARTTLS can be used even if the 'ssl' option is disabled. If you want to +# _force_ STARTTLS, you might need to disable 'ssl'. # # Default is yes. # @@ -732,7 +743,7 @@ # # It is best to leave this unset, in which case the correct version will be # automatically detected. In rare cases, it may be necessary to specify a -# particular version from: tls1, tls1_1, tls_1_2, ssl3, ssl23. +# particular version from: tls1, tls1_1, tls1_2, ssl3, ssl23. # # tls1_1 and tls1_2 are available with OpenSSL since v1.0.1. # @@ -846,15 +857,25 @@ # See below to learn how to get those. # # Specify the OAuth2 client id and secret to use for the connection.. -# Here's how to register an OAuth2 client for Gmail, as of 10-2-2016: -# - Go to the Google developer console -# https://console.developers.google.com/project -# - Create a new project +# Here's how to register an OAuth2 client for Gmail, as of 2017-05-15: +# - Go to the Gmail API overview console +# https://console.developers.google.com/apis/api/gmail.googleapis.com/overview +# - Create a new project, name doesn't matter, e.g. 'gmail-sync-bob' # - In API & Auth, select Credentials -# - Setup the OAuth Consent Screen -# - Then add Credentials of type OAuth 2.0 Client ID -# - Choose application type Other; type in a name for your client -# - You now have a client ID and client secret +# - Once created, click 'Enable' +# - Click 'Create credentials' in the enabled API overview + +# - In 'Add credentials to your project' select 'Gmail API' as the +# API type, and 'Other UI ...' (not 'Other non-UI ...') for +# 'Where will you be calling the API from?'. For 'What data will +# you be accessing?' select 'User data'. +# - Click 'What credentials do I need?' +# - Create an arbitrary 'Create an OAuth 2.0 client ID', +# e.g. 'gmail-sync-bob-client'. For 'Set up the OAuth 2.0 consent +# screen' select an arbitrary 'Product name shown to users', +# e.g. 'gmail-sync-bob-client' & click 'Continue'. +# - This gives you your client ID displayed on the screen. Click +# 'Download' to get a JSON file that also has the client secret. # #oauth2_client_id = YOUR_CLIENT_ID #oauth2_client_secret = YOUR_CLIENT_SECRET @@ -1060,7 +1081,8 @@ # download in UIDs order. # # If this is unset (the default), then up to maxconnections threads are used -# across all currently syncing folders. +# across all currently syncing folders. This option is sightly recommended in +# IMAP/IMAP mode (no maildir). # #singlethreadperfolder = no @@ -1206,7 +1228,7 @@ # This option stands in the [Repository RemoteExample] section. # # Propagate deletions from remote to local. Messages deleted in this repository -# won't get deleted on the local repositor if set to "no". Default is yes. +# won't get deleted on the local repository if set to "no". Default is yes. # # See sync_deletes in the LocalExample section, too. # @@ -1255,6 +1277,26 @@ #"cvlc --play-and-stop --play-and-exit /path/to/sound/file.mp3 > /dev/null 2>&1") +# This option stands in the [Repository RemoteExample] section. [TESTING] +# +# Operating under extreme network conditions (China) network connectivity +# especially for SSL can be so bad and so slow that absolutely every possible +# kind of connectivity error that can occur does occur. +# +# Rather than have offlineimap exit on errors it may be preferable to have it +# simply retry fetching of messages dozens of times. The alternative is to +# restart offlineimap in a constant loop, which may involve using significant +# CPU cycles (if the repository is large) to load up the UID database again. +# +# This option is best utilised in combination with socktimeout, which catches +# instances of interference by e.g. the GFW at the TCP layer, as well as China +# ISPs simply not coping. +# +# Default value: retrycount = 2 +# +#retrycount = 2 + + # This option stands in the [Repository RemoteExample] section. # # If offlineiamp is having troubles to download some UIDS, it's possible to get diff -Nru offlineimap-7.0.12+dfsg1/README.md offlineimap-7.1.2+dfsg1/README.md --- offlineimap-7.0.12+dfsg1/README.md 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/README.md 2017-07-10 14:58:50.000000000 +0000 @@ -15,6 +15,8 @@ OfflineIMAP is software that downloads your email mailbox(es) as **local Maildirs**. OfflineIMAP will synchronize both sides via *IMAP*. +## Why should I use OfflineIMAP? + IMAP's main downside is that you have to **trust** your email provider to not lose your email. While certainly unlikely, it's not impossible. With OfflineIMAP, you can download your Mailboxes and make you own backups of @@ -50,14 +52,6 @@ GNU General Public License v2. -## Why should I use OfflineIMAP? - -* It is **fast**. -* It is **reliable**. -* It is **flexible**. -* It is **safe**. - - ## Downloads You should first check if your distribution already packages OfflineIMAP for you. @@ -93,9 +87,9 @@ ## Requirements & dependencies * Python v2.7+ -* Python v3.4+ ***(experimental: [see known issues](https://github.com/OfflineIMAP/offlineimap/issues?q=is%3Aissue+is%3Aopen+label%3APy3))*** +* Python v3.4+ ***[STALLED] (experimental: [see known issues](https://github.com/OfflineIMAP/offlineimap/issues?q=is%3Aissue+is%3Aopen+label%3APy3))*** * six (required) -* imaplib2 >= 2.55 (optional) +* imaplib2 >= 2.57 (optional) ## Documentation diff -Nru offlineimap-7.0.12+dfsg1/scripts/get-repository.sh offlineimap-7.1.2+dfsg1/scripts/get-repository.sh --- offlineimap-7.0.12+dfsg1/scripts/get-repository.sh 2016-11-30 21:40:59.000000000 +0000 +++ offlineimap-7.1.2+dfsg1/scripts/get-repository.sh 2017-07-10 14:58:50.000000000 +0000 @@ -1,6 +1,6 @@ -#!/bin/bash +#!/bin/sh # -# Licence: this file is in the public deomain. +# Licence: this file is in the public domain. # # Download and configure the repositories of the website or wiki. @@ -10,7 +10,7 @@ # # TODO # -function final_note () { +final_note () { cat <