Ubuntu Software Center

Merge lp:~gary-lasker/software-center/catalog-published-date-lp803028 into lp:software-center

  1. catalog-published-date-lp803028
  2. Merge into trunk
Proposed by Gary Lasker
Status: Merged
Merged at revision: 2597
Proposed branch: lp:~gary-lasker/software-center/catalog-published-date-lp803028
Merge into: lp:software-center
Diff against target: 238 lines (+94/-16)
5 files modified
debian/changelog (+8/-1)
softwarecenter/db/application.py (+7/-1)
softwarecenter/db/update.py (+27/-3)
softwarecenter/enums.py (+1/-0)
test/test_database.py (+51/-11)
To merge this branch: bzr merge lp:~gary-lasker/software-center/catalog-published-date-lp803028
Reviewer Review Type Date Requested Status
Michael Vogt Pending
Review via email: mp+85401@code.launchpad.net

Description of the change

This branch implements the new 'date_published" field for for-purchase applications from the software-center-agent and incorporates it into catalog_times so that we display the for-purchase items in the What's New section correctly and in the correct order per their date of publication. This branch fixes bug 803028 and bug 86698.

The corresponding unit tests are added/updated. Note that currently the unit test points to the vpn test server because this functionality has not been deployed to the staging server yet. This is simply to prevent the unit tests failure before the staging server deployment is complete. I've marked this with a TODO to update it to point to the staging server once the support is there.

Finally, also note that I've left in the previous cataloged_time functionality as a fallback for for-purchase items so that these applications will not fall off the What's New list completely in the interim before the corresponding agent code is deployed on the production server. Once the agent is deployed, however, this fallback code will no longer be called and will have no effect. Nevertheless, I have marked it clearly with a TODO to remove it since it will serve no useful purpose after the production deployment is in place.

Should we also consider this fix as a candidate for an SRU to Oneiric?

Thanks!

To post a comment you must log in.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'debian/changelog'
2--- debian/changelog 2011-12-12 12:45:41 +0000
3+++ debian/changelog 2011-12-12 22:09:24 +0000
4@@ -15,7 +15,14 @@
5 - Add a translator comment to the 'Top Rated %s' string, name the
6 variable. LP: #868971
7
8- -- Michael Vogt <michael.vogt@ubuntu.com> Fri, 09 Dec 2011 18:26:59 +0100
9+ [ Gary Lasker ]
10+ * lp:~gary-lasker/software-center/catalog-published-date-lp803028:
11+ - add date_published value from the software-center-agent server
12+ for use with cataloged time so that for-purchase items appear
13+ correctly and in the proper order in What's New; include unit
14+ tests for the new functionality (LP: #803028, LP: #886698
15+
16+ -- Gary Lasker <gary.lasker@canonical.com> Mon, 12 Dec 2011 16:45:32 -0500
17
18 software-center (5.1.3.1) precise; urgency=low
19
20
21=== modified file 'softwarecenter/db/application.py'
22--- softwarecenter/db/application.py 2011-11-28 23:28:52 +0000
23+++ softwarecenter/db/application.py 2011-12-12 22:09:24 +0000
24@@ -339,7 +339,7 @@
25 from softwarecenter.db.history import get_pkg_history
26 self._history = get_pkg_history()
27 return self._history.get_installed_date(self.pkgname)
28-
29+
30 @property
31 def purchase_date(self):
32 if self._doc:
33@@ -359,6 +359,11 @@
34 return self._distro.get_license_text(self.component)
35
36 @property
37+ def date_published(self):
38+ if self._doc:
39+ return self._doc.get_value(XapianValues.DATE_PUBLISHED)
40+
41+ @property
42 def maintenance_status(self):
43 return self._distro.get_maintenance_status(
44 self._cache, self.display_name, self.pkgname, self.component,
45@@ -671,6 +676,7 @@
46 details.append(" license: %s" % self.license)
47 details.append(" license_key: %s" % self.license_key[0:3] + len(self.license_key)*"*")
48 details.append(" license_key_path: %s" % self.license_key_path)
49+ details.append(" date_published: %s" % self.date_published)
50 details.append(" maintenance_status: %s" % self.maintenance_status)
51 details.append(" pkg_state: %s" % self.pkg_state)
52 details.append(" price: %s" % self.price)
53
54=== modified file 'softwarecenter/db/update.py'
55--- softwarecenter/db/update.py 2011-12-12 09:15:52 +0000
56+++ softwarecenter/db/update.py 2011-12-12 22:09:24 +0000
57@@ -23,8 +23,8 @@
58 import json
59 import string
60 import shutil
61+import xapian
62 import time
63-import xapian
64
65 from gi.repository import GObject
66
67@@ -144,6 +144,7 @@
68 'Deb-Line' : 'deb_line',
69 'Signing-Key-Id' : 'signing_key_id',
70 'License' : 'license',
71+ 'Date-Published' : 'date_published',
72 'Purchased-Date' : 'purchase_date',
73 'License-Key' : 'license_key',
74 'License-Key-Path' : 'license_key_path',
75@@ -544,7 +545,7 @@
76 try:
77 # magic channel
78 entry.channel = AVAILABLE_FOR_PURCHASE_MAGIC_CHANNEL_NAME
79- # icon is transmited inline
80+ # icon is transmitted inline
81 if hasattr(entry, "icon_data") and entry.icon_data:
82 icondata = base64.b64decode(entry.icon_data)
83 elif hasattr(entry, "icon_64_data") and entry.icon_64_data:
84@@ -637,9 +638,15 @@
85 doc.add_value(axi_values["catalogedtime"],
86 xapian.sortable_serialise(cataloged_times[pkgname]))
87 else:
88- # also catalog apps not found in axi (e.g. for-purchase apps)
89+ #####################################################
90+ # TODO: This is just a fallback so that we keep our current
91+ # behavior of having new items for-purchase appear in
92+ # what's new...THIS SHOULD BE REMOVED after support
93+ # for date_purchased in the agent has been deployed
94+ # to the production server
95 doc.add_value(axi_values["catalogedtime"],
96 xapian.sortable_serialise(time.time()))
97+ #####################################################
98 # pocket (main, restricted, ...)
99 if parser.has_option_desktop("X-AppInstall-Section"):
100 archive_section = parser.get_desktop("X-AppInstall-Section")
101@@ -662,6 +669,23 @@
102 if parser.has_option_desktop("X-AppInstall-License"):
103 license = parser.get_desktop("X-AppInstall-License")
104 doc.add_value(XapianValues.LICENSE, license)
105+ # date published
106+ if parser.has_option_desktop("X-AppInstall-Date-Published"):
107+ date_published = parser.get_desktop("X-AppInstall-Date-Published")
108+ # strip the subseconds from the end of the published date string
109+ date_published = str(date_published).split(".")[0]
110+ doc.add_value(XapianValues.DATE_PUBLISHED,
111+ date_published)
112+ # we use the date published value for the cataloged time as well
113+ if "catalogedtime" in axi_values:
114+ LOG.debug(
115+ ("pkgname: %s, date_published cataloged time is: %s" %
116+ (pkgname, parser.get_desktop("date_published"))))
117+ date_published_sec = time.mktime(
118+ time.strptime(date_published,
119+ "%Y-%m-%d %H:%M:%S"))
120+ doc.add_value(axi_values["catalogedtime"],
121+ xapian.sortable_serialise(date_published_sec))
122 # purchased date
123 if parser.has_option_desktop("X-AppInstall-Purchased-Date"):
124 date = parser.get_desktop("X-AppInstall-Purchased-Date")
125
126=== modified file 'softwarecenter/enums.py'
127--- softwarecenter/enums.py 2011-11-11 03:05:45 +0000
128+++ softwarecenter/enums.py 2011-12-12 22:09:24 +0000
129@@ -129,6 +129,7 @@
130 LICENSE_KEY_PATH = 193 # no longer used
131 LICENSE = 194
132 VIDEO_URL = 195
133+ DATE_PUBLISHED = 196
134
135 # fake channels
136 PURCHASED_NEEDS_REINSTALL_MAGIC_CHANNEL_NAME = "for-pay-needs-reinstall"
137
138=== modified file 'test/test_database.py'
139--- test/test_database.py 2011-10-07 14:48:05 +0000
140+++ test/test_database.py 2011-12-12 22:09:24 +0000
141@@ -15,7 +15,10 @@
142 from softwarecenter.db.enquire import AppEnquire
143 from softwarecenter.db.database import parse_axi_values_file
144 from softwarecenter.db.pkginfo import get_pkg_info
145-from softwarecenter.db.update import update_from_app_install_data, update_from_var_lib_apt_lists, update_from_appstream_xml
146+from softwarecenter.db.update import (update_from_app_install_data,
147+ update_from_var_lib_apt_lists,
148+ update_from_appstream_xml,
149+ update_from_software_center_agent)
150 from softwarecenter.enums import (
151 XapianValues,
152 PkgStates,
153@@ -94,7 +97,6 @@
154 self.assertEqual(db.get_doccount(), 1)
155
156 def test_build_from_software_center_agent(self):
157- from softwarecenter.db.update import update_from_software_center_agent
158 db = xapian.WritableDatabase("./data/test.db",
159 xapian.DB_CREATE_OR_OVERWRITE)
160 cache = apt.Cache()
161@@ -120,7 +122,6 @@
162 doc.get_value(XapianValues.ICON).startswith("sc-agent"))
163
164 def test_license_string_data_from_software_center_agent(self):
165- from softwarecenter.db.update import update_from_software_center_agent
166 from softwarecenter.testutils import get_test_pkg_info
167 #os.environ["SOFTWARE_CENTER_DEBUG_HTTP"] = "1"
168 os.environ["SOFTWARE_CENTER_BUY_HOST"] = "http://sc.staging.ubuntu.com/"
169@@ -279,9 +280,45 @@
170 doc.get_value(value_time) >= last_time
171 last_time = doc.get_value(value_time)
172
173- def test_non_axi_apps_cataloged_time(self):
174- db = xapian.WritableDatabase("./data/test.db",
175- xapian.DB_CREATE_OR_OVERWRITE)
176+ def test_for_purchase_apps_date_published(self):
177+ from softwarecenter.testutils import get_test_pkg_info
178+ #os.environ["SOFTWARE_CENTER_DEBUG_HTTP"] = "1"
179+ ####################################################################
180+ # TODO: Point the following back to http://sc.staging.ubuntu.com/ once
181+ # the support for published date is deployed there
182+ ####################################################################
183+ os.environ["SOFTWARE_CENTER_BUY_HOST"] = "http://sca.razorgirl.info"
184+ # staging does not have a valid cert
185+ os.environ["PISTON_MINI_CLIENT_DISABLE_SSL_VALIDATION"] = "1"
186+ cache = get_test_pkg_info()
187+ db = xapian.WritableDatabase("./data/test.db",
188+ xapian.DB_CREATE_OR_OVERWRITE)
189+ res = update_from_software_center_agent(db, cache, ignore_cache=True)
190+ self.assertTrue(res)
191+
192+ for p in db.postlist(""):
193+ doc = db.get_document(p.docid)
194+ date_published = doc.get_value(XapianValues.DATE_PUBLISHED)
195+ # make sure that a date_published value is provided
196+ self.assertNotEqual(date_published, "")
197+ self.assertNotEqual(date_published, None)
198+ del os.environ["SOFTWARE_CENTER_BUY_HOST"]
199+
200+ def test_for_purchase_apps_cataloged_time(self):
201+ from softwarecenter.testutils import get_test_pkg_info
202+ #os.environ["SOFTWARE_CENTER_DEBUG_HTTP"] = "1"
203+ ####################################################################
204+ # TODO: Point the following back to http://sc.staging.ubuntu.com/ once
205+ # the support for published date is deployed there
206+ ####################################################################
207+ os.environ["SOFTWARE_CENTER_BUY_HOST"] = "http://sca.razorgirl.info"
208+ # staging does not have a valid cert
209+ os.environ["PISTON_MINI_CLIENT_DISABLE_SSL_VALIDATION"] = "1"
210+ cache = get_test_pkg_info()
211+ db = xapian.WritableDatabase("./data/test.db",
212+ xapian.DB_CREATE_OR_OVERWRITE)
213+ res = update_from_software_center_agent(db, cache, ignore_cache=True)
214+ self.assertTrue(res)
215 res = update_from_app_install_data(db, self.cache, datadir="./data/desktop")
216 self.assertTrue(res)
217 db = StoreDatabase("./data/test.db", self.cache)
218@@ -291,12 +328,15 @@
219 sc_app = Application("Ubuntu Software Center Test", "software-center")
220 sc_doc = db.get_xapian_document(sc_app.appname, sc_app.pkgname)
221 sc_cataloged_time = sc_doc.get_value(axi_value_time)
222- so_app = Application("Scintillant Orange", "scintillant-orange")
223- so_doc = db.get_xapian_document(so_app.appname, so_app.pkgname)
224- so_cataloged_time = so_doc.get_value(axi_value_time)
225- # the test package Scintillant Orange should be cataloged at a
226+ for_purch_app = Application("For Purchase Test App", "hellox")
227+ for_purch_doc = db.get_xapian_document(for_purch_app.appname,
228+ for_purch_app.pkgname)
229+ for_purch_cataloged_time = for_purch_doc.get_value(axi_value_time)
230+ # the for-purchase test package should be cataloged at a
231 # later time than axi package Ubuntu Software Center
232- self.assertTrue(so_cataloged_time > sc_cataloged_time)
233+ self.assertTrue(for_purch_cataloged_time > sc_cataloged_time)
234+
235+ del os.environ["SOFTWARE_CENTER_BUY_HOST"]
236
237 def test_parse_axi_values_file(self):
238 s = """

Subscribers

People subscribed via source and target branches