Keryx Tutorial: Bringing Updates Home

Update 20/2/09: Keryx 0.92 has been released, with several new features and a few important bug fixes. If you are using the previous version, please upgrade now. Check out the release notes for a list of what has changed. For a screenshot walk-through of the changes, check out this blog post.

Though in most North American cities one cannot find a spot without at least a weak WiFi signal, many of us Linux geeks still live in rural areas with less Internet connectivity. Also, in various non-Westernized nations, there is a growing number of Linux users who may have a computer at home, but cannot afford a decent connection. For both groups, software updates typically demand an Internet connection, which can make updating difficult if not impossible. There is now a solution though, a new program called Keryx.

Keryx was written by Southern Illinois University computer science student Chris Oliver, who wanted a way  to download software and updates for Ubuntu systems that had little or no connectivity. Simply put Keryx on your pen drive, use it to create a new project file which retains a copy of your software sources and other system details, then take the pen drive to a computer with a better connection. Via it’s Synaptic like interface, users can then select all updates for download, plus select any other software they may want to install, complete with dependency resolution.

Because it is written in Python, and utilizes wxWidgets for it’s interface, Keryx can run on Linux, OSX and Windows. Pre-compiled binaries for Windows are included in the download (meaning you don’t need to install Python and wxWidgets first), and similar binaries for OSX and Linux are in the development road-map, along with Debian/Ubuntu packages.

Using Keryx.

This tutorial will walk you through the simple process of using Keryx to get updates and new software. Keryx currently only works for Debian based distros, but there are plans for adding support for a number of other package management systems. The system being updated is running Ubuntu 8.10, with no network connectivity. The system that will be grabbing the updates is running Windows XP, though it could just as easily be Windows 95 through Vista, OSX, or another Linux box.

Step #1: Download Keryx.

Extract the zip file
Go to the Keryx website and click the download link. Once the download is complete, put it on a USB pen drive that has a decent amount of free space, and unzip it.

Step #2: Create a project.

Starting a project from the CLI
Keryx uses wxWidgets for it’s graphical interface, and a default Ubuntu install does not have wxWidgets installed. Therefore you must create your project file in a terminal window. Fear not, as it is really quite quick and painless.

Simply open up your terminal, and then navigate into the “linux” directory inside the Keryx folder. On my computer this was “/media/disk/keryx-0.91/linux” but it will look a little different for you, depending upon what your pen drive is called. Once you are in that directory, enter in the following, making sure to replace <project> for whatever you want to call your project and <plugin> for they type of system you are updating, in this case debian.

python keryx.py –create <project> <plugin>

note: the above is two dashes, but my font makes it look like one.

In a few moments the project will be made. When this happens, close out of the terminal and safely remove your pen drive, to take to another computer.

Note: For some people, Keryx may experience difficulty fetching the package list files in the next step, if their repository mirror is set to the regional default. If you experience this, consider selecting a different mirror in System/Administration/Software_Sources and then repeating step #2.

Step #3: Opening Keryx and downloading the package list.

In this tutorial I am using a computer running Windows XP as the computer with a high speed connection. However, this could just as easily be done on any Linux or OSX computer, so long as they both had Python and wxWidgets installed.

The Keryx main window
On your Windows computer, plug in the pen drive and open up the Keryx folder. In this folder you will see a “win32″ folder containing prepackaged binaries for Keryx. Using these you can run Keryx without having to first install Python and wxWidgets, making Keryx a very portable application. The file you need to run is called “keryx.exe,” though Windows may hide the .exe part from you. When Keryx opens, click “Open Project” and find the project file you created a few minutes ago.

Downloading package lists
When you open Keryx, go ahead and let it download the latest package list.

Step #4: Download updates.

The package list
Once the package lists have been downloaded, you’ll see the full Synaptic-like package list in Keryx. This list can be sorted by package name, status (not installed, installed, needs updating, etc.), etc. The first thing you’ll want to do is click “Get Updates” near the top of the window.

Snarfing 210 updates...
When I started the download, Keryx had 210 files to download. Your number will vary, but you are likely to have a lot of updates if you have a fresh install. Keryx will tell you when it is finished, so you might want to go get some coffee.

Step #5: Download wxWidgets.

Searching for a package
You probably want to be able to install wxWidgets on your Ubuntu system, so that you can run the Keryx graphical interface on it. Near the top of the window, start typing “wxversion” in the search field. Because the Keryx package search tool is so amazingly fast, you’ll only need to type the first few letters before you see python-wxversion. Right-click this package and then click download. It has several dependencies that Keryx will tell you about, so go ahead and let the program download those as well.

Step #6: Install your packages.

Installing updates via dpkg
With all your packages downloaded, close Keryx, safely remove your pen drive, and go back to your Ubuntu machine. When you plug the pen drive back into your Ubuntu machine, you’ll notice that the package are stored in projects/<project>/packages (where <project> is the name you gave the project). You’ll need to open up your terminal again, and navigate to this directory. Once there, run the following:

sudo dpkg -i –force-depends *.deb

Running updates
This line will install and/or update all the packages in that directory. The “force-depends” parameter is necessary in this case, as we are installing the wxWidgets packages, which have a circular dependency. Without this parameter, dpkg will start whining at you. If you have as many packages as I had, this may take a while. Go refill your coffee, and by the time you get back, the install might be done. Thats all there is to it!

What is next for Keryx?.

Despite already being a rather useful tool, there are a lot of enhancements planed for the near future with Keryx. Most of it’s features, including it’s package management support, are implemented via a very flexible plugin infrastructure. So if you are a Python hacker and would like to help implement some of the new features, or have some ideas of your own, feel free to check out the code and dive in!

If you run into any problems using Keryx, or would like to report a bug, check out the friendly forums.

Upcoming features.

  • Support for more distributions, such as Fedora, Red Hat, Mandriva et al.
  • Package management like support for downloading and installing useful Open Source Windows software.
  • Built-in installation of downloaded packages, so the user need not mess with the CLI
  • Pre-compiled self contained binaries for Linux and OSX (like what already exists in the win32 folder), so that no matter what OS a user is running, they will be able to simply plug in their USB drive and run the graphical interface.
  • Improved documentation

38 Responses to “Keryx Tutorial: Bringing Updates Home”

Chris Oliver's Blog » Keryx 0.91 Released on January 4th, 2009 8:53 pm:

[...] grab all 200+ updates onto a flash drive so that he could update his machine. You can view the post excid3 (at) gmail (dot) com. A friend of mine recently tried out the newest version on a fresh install of Ubuntu. He had never [...]


Chris Oliver's Blog » Keryx 0.91 on January 4th, 2009 9:02 pm:

[...] grab all 200+ updates onto a flash drive so that he could update his machine. You can view the post excid3 (at) gmail (dot) com. A friend of mine recently tried out the newest version on a fresh install of Ubuntu. He had never [...]


tracyanne on January 5th, 2009 3:52 pm:

I’d be very interested in a version for Mandriva, or help making the changes.


Douglass Clem on January 5th, 2009 3:57 pm:

I think Mandriva support would be great! Feel free to register and start a thread about it on the forums. Alternately, I’m sure the main developer would be glad to hear from you if you send a message to excid3 (at) gmail (d0t) com.


Chris Oliver on January 5th, 2009 7:17 pm:

I would be very interested in helping you build a Mandriva plugin. Shoot me an email or drop by the forums or check out my IM usernames on my blog (About Me page). I think it would be a good jumping off point to get the community involved in working on extra plugins for Keryx.


Boycott Novell » Links 05/01/2009: GNU/Linux at Honda Dealership, KDE 4.2 Release Parties Planned on January 5th, 2009 8:41 pm:

[...] Keryx Tutorial: Bringing Updates Home Though in most North American cities one cannot find a spot without at least a weak WiFi signal, many of us Linux geeks still live in rural areas with less Internet connectivity. Also, in various non-Westernized nations, there is a growing number of Linux users who may have a computer at home, but cannot afford a decent connection. For both groups, software updates typically demand an Internet connection, which can make updating difficult if not impossible. There is now a solution though, a new program called Keryx. [...]


Fazendo upgrade a Debian e Ubuntu e familiares sem banda larga « O Vigia on January 6th, 2009 3:40 am:

[...] Dêem olhada [...]


Rahim on January 6th, 2009 7:15 am:

Nice! Thank you, there are still a lot of people who don’t have broadband, and this has been greatly needed for a long time.


Chris Oliver on January 6th, 2009 5:11 pm:

I apologize for the downtime on the Keryx homepage, turns out the web admin had a bit of trouble installing our new wiki software and accidentally took down the homepage with it. Everything should be back up just fine.


Jay on January 8th, 2009 5:03 pm:

Douglass Clem said: “…in various non-Westernized nations….”

Doug, thanks for the tutorial. I’d just like to point out that having advanced technological infrastructure is not the same as being “Westernized”.


Douglass Clem on January 8th, 2009 6:28 pm:

…having advanced technological infrastructure is not the same as being “Westernized”.

I completely agree with that statement. While drafting the tutorial, I spent several minutes trying to find a better way to phrase that sentence, so that it could not be mistaken for ethnocentricity. I did not discover a better way to phrase it in that few minutes, so I moved on.

Indeed, I can think of several countries off the top of my head (S. Korea, Japan) that are far Eastern countries and far surpass the USA in terms of broadband capacity and availability. Also, a number of South American countries would be a good example of countries that are “Western” in a geographical sense, but have poor connectivity. However, though there are some notable exceptions, such as the ones given above, there is a pronounced inequality of connectivity, and the countries that fit into the “Western European” category in a cultural sense (and the USA does fit into that category, despite it’s position across the Atlantic) have on average more connectivity than nations outside of that category. It was in this sense that I used the phrase “non-Westernized.”


sparrowt on January 14th, 2009 7:08 am:

i love u guys, keryx is just what ive been looking for, cos i use ubuntu8.04 at home, but have no internet on it, so have to go through downloading all dependencies separately!
no longer loovvv you all!!!


Gareth on February 1st, 2009 6:51 am:

I’m being told that after typing in:

python keryx.py -c testytest

That:

Search.py failed to load
Example.py failed to load


Chris Oliver on February 1st, 2009 1:36 pm:

Those packages require that the wxPython are installed. They will fail to load because you do not have wxPython installed. It is supposed to happen, however I will patch it to give a reason as to why they were not loaded in the upcoming releases. Thanks for reminding me to fix it! :)


brijith on February 3rd, 2009 3:13 am:

When I run python keryx.py -c macs I got the following error.

/Desktop/keryx-0.91/linux$ python keryx.py -c macs
Traceback (most recent call last):
File “keryx.py”, line 21, in ?
from lib import consts, log, project
File “/home/brijith/Desktop/keryx-0.91/linux/lib/project.py”, line 28
class Project():
^
SyntaxError: invalid syntax


Chris Oliver on February 3rd, 2009 6:19 am:

@brijith thats interesting. no one has ever had an issue like that so far. you did not edit project.py did you? i would try downloading it again and running it. if that doesnt work, copy and paste me the contents of project.py and i will have a look and see what is causing the problem. hopefully its something simple! :P


Victor on February 11th, 2009 11:26 pm:

To access internet I connect through a proxy and need to set the proxy IP number and port. Can’t figure out how to do that. (Or, have I missed out on something?) Any help please ?


excid3 on February 12th, 2009 1:43 pm:

Actually, Keryx does not support proxy connections just yet. This is something that I thought I would address when the issue came up and it has not until you mentioned it. I thought about it a while ago and have forgotten about it since. The downloading in Keryx is kind of shoddy as it is and could use a revamp anyways. I will make this my next task to implement so that you and anyone else who require proxies can use Keryx properly.


Dick Stubbs on February 12th, 2009 4:12 pm:

Looks Like a great program. I’d really
like to try it as I am one of the Linux geeks
described earlier in the article who lives in
a rural area without good internet service.
I was so excited to try this program, I
re-installed Ubuntu 8.10 this morning on my
home computer, then created the project file
on the new install. When I got to work this
afternoon, I plugged in my flashdrive to my
windows XP box and attempted to execute the
‘keryx.exe’ file from the win32 subfolder.
The program crashed with the error
‘This application has failed to start because
MSVCP71.dll was not found. Re-installing the
application may fix this problem.
Below is the contents of the keryx.exe.log file:
Traceback (most recent call last):
File “keryx.py”, line 91, in
File “lib\wxkeryx\__init__.pyo”, line 19, in
File “wx\__init__.pyo”, line 45, in
File “wx\_core.pyo”, line 4, in
File “wx\_core_.pyo”, line 12, in
File “wx\_core_.pyo”, line 10, in __load
ImportError: DLL load failed: The specified module could not be found.

Any ideas?


Douglass Clem on February 12th, 2009 4:28 pm:

I emailed your comment on to Chris, the main developer on the Keryx project. After a bit of googling, I found that there are a number of programs that get this same error message. You can download MSVCP71.dll from the link below. From what I’ve read, this usually fixes it. Give that a try, then come back to this comment thread to tell us if it worked or not.

http://www.dll-files.com/dllindex/dll-files.shtml?msvcp71


Dick Stubbs on February 12th, 2009 5:04 pm:

Worked perfect! Thanks so much. This is
just the program I’ve been looking for.


excid3 on February 12th, 2009 10:02 pm:

As Douglass mentioned, the MSVCP71.DLL error is easily fixable by copying the C++ runtime dll to the Keryx directory. I have documented this issue and fix on the forums: http://keryx.betaserver.org/forum/index.php/topic,39.msg231.html#msg231


excid3 on February 20th, 2009 12:55 am:

Just wanted to announce Keryx 0.92’s release today. Lots of new features making it easier to use than ever! Hope you like it!


Corniger on February 20th, 2009 5:51 am:

Hi! That’s a rockin’ cool piece of software. I can’t manage to get my WLan working in intrepid, so at least I see a chance now. I’m getting a bunch of errors and Ubuntu isn’t satisfied, so I’m re-running the whole procedure several times and see what comes out :) I don’t know anything about Ubuntu, and, to be honest, in the moment I really dont WANT to, I just wanna run Blender 3d on it and other Open Source CG software (64bits!), so I don’t really know what I’m doing and I don’t have the time or competence. And Keryx seems to be saving me :)
I just can’t create a new project under Linux anymore – though I don’t nkow if that’s important to get more updates…


excid3 on February 24th, 2009 2:39 am:

Crashsystems: The -c parameter no longer exists. It is only –create now. I figured this would give more description to what they do and not be left with some unexplained command-line parameters which are easily forgotten.

Corniger: Are you using 0.92 or 0.91? if you already have a project created you can just copy your /var/lib/dpkg/status file to your projects folder and replace the status file that is in there already. That should update your project with the most recently installed applications so that those packages wont be installed again.

Feel free to email me or join the Keryx forums if you need any help. http://keryxproject.org/forum


Update Ubuntu on a machine without an Internet Connection at andylockran’s blog on February 25th, 2009 7:14 am:

[...] have one where I’ll be living – I’m not yet sure. However, having read a little about Keryx – I’m pretty sure I’ll have no issues.  It looks to be a very clever way of updating [...]


Chamila on March 5th, 2009 9:40 pm:

hi,

i downloaded the zip file and extracted it into my flash. but when i tried to create a project in the terminal this error was given.

Loading config: /media/SHAPESHIFTR/keryx-0.92/linux/keryx.conf
Traceback (most recent call last):
File “keryx.py”, line 105, in
import lib.wxkeryx
File “/media/SHAPESHIFTR/keryx-0.92/linux/lib/wxkeryx/__init__.py”, line 19, in
import wx
ImportError: No module named wx

tried searching the net. but couldn’t find anything related to keryx. as i’m not a python programmer i haven’t used python before. i’m using 0.92 version of keryx.

i understand that it needs wx-version but i’m trying it on a fresh intrepid installation. assume no internet connection.


Goulou on March 26th, 2009 4:07 am:

A very good tool indeed!

Maybe you could add a (not necessary) feature : once you get back to the offline computer, add a button “install updates” to avoid getting to the console.

Also, could you possibly add statistics (especially before download to know how much we have to download), and a better progress bar that tells us at least roughly how much time we should wait?

Congratulation for your work, it is actually very usefull and working very well!


Douglass Clem on March 26th, 2009 9:07 am:

Our thoughts exactly! We have plans for automatic installation, and hopefully it won’t be too long before they are implemented. Basically, Keryx is going to add an entry into your sources.list to create a local (on the USB stick) repository of the packages you downloaded, allowing Keryx to hook into apt-get.

We have also been planning on improving the progress bar, and the statistics idea (how many packages being installed, etc) is a very good idea. I’m not sure how feasible the installation time estimate is though, as that his highly dependent upon the hardware of the machine running the installations.


Goulou on March 27th, 2009 4:34 am:

that’s some good news :-)

In my case, what was most important was not to have an estimation of installation time, but rather an estimation of download time, which is easier to estimate after the first few seconds…
For installation, maybe you could just add something like “installing package X of Y”, or make an estimation based upon the size of the packages themselves… (ok, it’s not very accurate, but it’s usually quite verified…)


Sudhir on April 22nd, 2009 12:41 am:

Thanks you guyz..
This is what I was looking for since a long time.

However I have a question, whenever I click on get updates, it downloads hundreds of files. does it download all the new softwares from repository or just updates for installed packages?

Thanks


Chiflado on April 24th, 2009 4:11 pm:

So how do you use Keryx to download updates with OS X? That would help a lot, I’ve got a Linux friend with no internet at home in need of updates

Thanks


Apoorv Parle on April 29th, 2009 3:43 am:

This is great. Just great.

I have a suggestion. If I want to download 2 softwares X and Y but I want to install only X and if X doesn’t work then I’ll install Y.

Why don’t you just copy the package lists to /var/lib/apt/lists and deb files to the /var/cache/apt/archives so that installation can be done by apt-get or by synaptic or by any other tool like restricted driver manager etc.
This will let the beginners and GUI lovers use Synaptic and will make your tool usable with restricted-driver-manager etc


Chris Oliver on April 29th, 2009 9:34 am:

@Sudhir
Keryx downloads the updates for the installed packages that you have when you select Get Updates. This is like using Update Manager on your offline machine. It downloads the list of all the latest software to get you the newest updates and also lists the newest packages so that you can go download the newest software as well, for example, the latest VLC would be available if you used Keryx.

@Chiflado
For now you need python installed as well as wxPython (http://www.wxpython.org/download.php). You can then run Keryx from keryx.py and if you’ve got wxPython installed it will bring up the interface just like it would if you ran it on Windows or on Linux that had wxPython installed.

@Apoorv
Since Keryx is still in development I have just been able to finish up the bare minimum, it does the absolute basics. Over the summer that is going to be one of the main features I will be adding. :) It will take care of installing the packages, copying over the list files and the packages over to the apt cache folder. I’ll try to keep everyone posted on the development/release status!


Renaud on May 14th, 2009 1:02 am:

Hi,

Keryx is quite a very nice piece of software. Congratulations!

Playing a bit with it I noticed the following things:

- I ran the ‘Get updates’ procedure twice in a row and it looks like the download is performed at each time, even if all the updates were successfully downloaded during the first run. Does the update procedure check if the updates are already present in the project directory? Maybe, this could avoid some extra downloads and save time.

- some repositories of my sources list contain the same packages but with different versions (eg. Ubuntu official repos with some version of R and CRAN repository with the latest version of R, usually newer than the Ubuntu one).
The packages installed on my remote machine are the latest ones (from CRAN). When I run Keryx to get the updates, all these packages appear in red, flaged as ‘Newer than Repository’. I tried putting the CRAN repository at the top and end of the sources list but it did not change anything.
Is it a known behaviour? Does the list loader take into account multiple available versions of packages?

Thanks and well done again.


Chris Oliver on June 7th, 2009 10:57 am:

Currently Keryx uses a VERY simplistic download system. It does not work with proxies very well and does not check to see if files already exist. This is going to be changed in Keryx 1.0 though, we will be using Urlgrabber which was created for Yum. It works extremely well and will resolve all of our downloading issues like that.

The package lists should only download the latest packages when you select to install something. The version comparison is actually semi-complicated and could very well be flawed and that would be what is causing your problem. This should all be resolved in the next version of Keryx as well. Remember, its still pretty much an Alpha.


VoltRabbit on June 7th, 2009 1:14 pm:

I signed onto the members site, but this topic seems to be more active at the moment.

The tutorial looks great, but as a classic user, I am hitting some snags. I get as far as step 2. Ive been doing some reading trying figure it out, maybe I need the wxPython thing.

I run /media/kingston/keryx-0.92.1/linux and get:
Bash: /media/kingston/keryx-0.92.1/linux: is a directory

user@user-desktop:-$ python keryx.py –create jake-desktop debian
python: can’t open file ‘keryx.py’: [Errno 2] No such file or directory.

I downloaded keryx 0.92.1 on my kingston thumbdrive while using my windows xp online pc. I have tried removing the extracted files I originally extracted in while using the windows pc, and extracting the files while on my ubuntu offline pc.

Cool program, can’t wait to figure it out, my bad.


Chris Oliver on June 7th, 2009 1:45 pm:

@VoltRabbit
Issue resolved here: http://keryxproject.org/forums/index.php?page=findpost&post=18

You just forgot to use “cd” in front of the keryx directory. :)