Wednesday September 21, 2011

Magento: Google Checkout Button in Side Cart

I haven't written any articles in a while, and thought I should write a quick one about a Magento 'fix' I needed to do today.  Magento isn't the cleanest and most straight-forward software to extend, so I usually find myself doing a quick search to see if anyone has already tackled whatever customization a customer is asking for.  This time, I only found bad examples of how to do this.

The best way to extend Magento in order to future-proof your modifications is to define overrides in a local.xml layout and doing everything you possibly can to avoid hacking up or overriding any default layouts or templates.  Adding the Google Checkout button to the side bar cart is a typical example where I found several people outlining how to hack up existing templates to accomplish this.

None of this is necessary.

Following is the local.xml modifications that will insert Google Checkout into the cart side bar.

<!--?xml version="1.0" encoding="UTF-8"?-->
<layout>
  <default>
    <!-- Add google checkout link in side cart -->
    <reference name="cart_sidebar.extra_actions">
      <block type="googlecheckout/link" name="googlecheckout.partner.cart_sidebar.shortcut" template="googlecheckout/link.phtml" before="-">
    </block></reference>
  </default>
</layout>

Please make sure you comment what your modifications to local.xml do. It's not always intuitive months later, and I might have to work on it after you do. :)

Sunday July 25, 2010

Rackspace Cloud Hosting

After reciting the same facts and figures for numerous customers countless times (and no doubt forgetting something each time!), I've finally taken the time to put up a page dedicated to why we choose Rackspace Cloud hosting. I'm sure we'll be improving upon it as time goes on, but I'm glad we finally have this information in one place since we deal with so many people trying to migrated to cloud hosting to reduce expenses.

Why The Rackspace Cloud?

Tuesday July 07, 2009

OpenSolaris rpool sharing disks

We've been using Opensolaris (and Solaris) more and more; the advantages in technical prowess outweigh the few advantages Linux has left in ease of use. One thing we've been frustrated with a couple of times is the apparent inability to to create custom slice layouts on root disks during installation.

We're building a 16 drive, 16TB ZFS archival server, and we didn't want to waste two drives (or cram two additional drives in the chassis) for root volumes when we had plenty of space on our 16 data drives.  We wanted to run a 50GB root partition across one ZFS raidz1 group. I'm hoping there's a simpler way to do this, and we just lacked the patience for searching for it. In the event that there isn't, we decided to throw an outline of the procedure we used up here.

1. Partition a system disk with a 50GB Solaris partition.

2. Install Solaris.

3. Partition a second system disk with the Solaris partition as the entire disk.

4. Use prtvtoc | fmthard to copy the layout from disk 1 to disk 2.

5. Expand slice 2 on disk 2 to consume the entire disk.

6. ZFS attach disk 2 to disk 1.

7. Use installgrub to place grub on disk 2.

8. Boot off disk 2.

9. Detach disk 1 from rpool.

10. Partition disk 1 like disk 2

11. ZFS attach disk 1 to disk 2

12. Use installgrub on disk 1

13. Create a new slice on both disks for the raidz group

13. Repeat steps 10-13 for each disk you plan to use in the raidz group

14. Create a raidz group using the new slice on each of the disks

It's a little bit involved, but in 20 minutes we created a four-way root mirror and had a 4 x 900GB raidz partition on the rest of that disk group.  This saved us the hassle of shoe-horning and cabling two system drives, reduced potential points of failure, and dropped our power  consumption slightly over what we expect will be a 5+ year installation.

If there's a simpler way to do this, please comment. :)

Thursday June 25, 2009

Migrating and filtering Subversion repositories

I've been tasked with merging one subversion repository into another. The source repository is 35GB and the destination about 2GB.  The source repository is bloated by the ritual inclusion of large directories of binary files that were wiped and re-added continually as well as branched and tagged hundreds of times.

The goal is to eliminate these binaries and all history related to them in order to reduce the repository down to a much more manageable size.  Easy enough to do with svnadmin dump and svndumpfilter exclude X,Y,Z, right?

I was pleased to find out that svndumpfilter does take multiple excludes or includes on the command line. I didn't see this in the documentation, and found a surprising number of people piping multiple copies of svndumpfilter together in order to exclude multiple things.  I figured out that I'd need 1,188 exclusions in order to wipe out all permutations of these binary objects in trunk, tags, and branches.  It was simple enough to build a script with awk that would list the exclusions on the svndumpfilter command line.

Unfortunately, 162 revisions into 45,000, it failed because of an svn copy from an excluded part into an included part. Svndumpfilter cannot handle this scenario because by the time it identifies that it needs something it's excluded, it's already well past it in the stream.  I was half expecting this from the documentation, but hoping against reason that none of the developes had ever copied anything. :)

More searching turned up svndumpfilter2 and svndumpfilter3 - two scripts that were written to solve this specific problem.  They use svnlook to grab the missing piece directly from the repository. Unfortunately, I quickly found that svndumpfilter is known for crashing on large repositories - of a whopping 150MB.  Svndumpfilter3 was written to overcome svndumpfilter2's limitations, but the author's web page has a large disclaimer that it's known to fail on large repositories - not confidence inspiring.

Browsing through the subversion 1.5 manual again, I found this blurb that caused a lightbulb to go off:

Q: How does svnsync deal with parts of the master repository that I'm not
   authorized to read?

A: svnsync will simply not copy parts of the repository that you
   cannot read; files copied from "private" parts of the repository
   into "public" parts will look as if they have been added from
   scratch.  If a revision only modifies files that you cannot read,
   it will appear to be empty.  (Just like with "svn log", log
   messages from revisions you cannot read part of will be empty.)

This might be the exact functionality we need -  it would give us the opportunity to use svnsync to keep a synchronized mirror going up to the point of the final split while easily filtering out the data we didn't want.  Since I haven't stumbled across this filtering technique described elsewhere on the net, I figured it might be blog-worthy. Svnsync doesn't support synchronizing multiple subdirectories to a single repository, so we used a temporary repository as a mirror, dumped it, and then imported it into our final repository.

Outline:

1. Enable authorization in repository/conf/svnserve.conf
2. Configure authz to restrict access to the directories you want to exclude
3. Create a destination repository for svnsync
4. Use svnsync to synchronize to the temporary destination repository
   
5. Dump the destination repository
6. Import the dump into the destination repository
7. Verify

1. Enable authorization in repository/conf/snvserve.conf

This will differ if you're using webdav, but in this particular case, ssh+svnserve is the protocol of choice.

### Uncomment the line below to use the default authorization file
authz-db = authz

2. Configure authz to restrict access to the directories you want to exclude

You'll need to modify the authz file to restrict access to the paths in the source repository that you do not want to get synchronized to your destination repository. This is easily accomplished by setting the permissions for the user you're going to use to do the svnsync to nothing - that is, no "r" or "rw."

[/]
* = rw

[/trunk/binaryfilespath]
svnsync =

[/tags]
svnsync  =

[/branches]
svnsync =

3. Create a destination repository for svnsync

You need a new repository for svnsync to mirror to, and you'll have to modify the pre-revprop-change script so that it allows at least the user you're using to modify revprops.

$ svnadmin create dest
$ cat <<'EOF' > dest/hooks/pre-revprop-change
#!/bin/sh
exit 0
EOF

4. Use svnsync to synchronize to the temporary destination repository

One nice advantage of using svnsync is that you can keep running the
sync to keep it updated and allow for a very small delta at the time of
switchover.  We used this to run the initial sync hours ahead of time,
and only spend a few minutes at the time all the developers had to log
out of the system.  Note: You only run initialize once!

$ svnsync init --username svnsync file://`pwd`/dest svn+ssh://svnhost/var/svn/oldrepository
Copied properties for revision 0
$ svnsync sync file://`pwd`/dest
Committed revision 1.
Copied properties for revision 1.
Committed revision 2.
Copied properties for revision 2.
Committed revision 3.
Copied properties for revision 3.
...
$ svnsync sync file://`pwd`/dest
Committed revision 47483.
Copied properties for revision 47483.
Committed revision 47484.
Copied properties for revision 47484.
...

5. Dump the destination repository

$ svnadmin dump ./dest > clean-repository.dump

6. Import the dump into the destination repository

This step is a tad bit trickier than you might suspect. If you want to import the dump into a new subdirectory in the repository, you must create that subdirectory first or you'll get an error:

svnadmin: File not found: transaction '0-1', path 'trunk/newdir'

$ mkdir -p trunk/newdir
$ svn import -m "Import structure" trunk file:///var/svn/repository
$ svnadmin load --parent-dir trunk/newdir /var/svn/repository < clean-repository.dump
<<< Started new transaction, based on original revision 1

------- Committed new rev 44888 (loaded from original rev 1) >>>

7. Verify

It goes without saying, test your newly imported repository. You don't want to get 3 months down the line and realize you missed a truckload of revision history.

Monday February 23, 2009

Fun with OpenSolaris snv_107 and Nvidia drivers

I've been eagerly waiting an upgrade from OpenSolaris 2008.11 that would provide working WPA2 PSK for my laptop, as my laptop is the easiest system for operating system experimentation. I've heard stories of success with snv_107, so I decided to give it a shot.

Somehow the upgrade from snv_101b to snv_107 was quirky - Xorg didn't work and 'pkg fix' showed such a large number of problems while crashing repeatedly. I decided a fresh installation was called for.

One of the first things I do on new Linux or Solaris workstations is install the closed-source Nvidia drivers. On OpenSolaris, as opposed to my usual OS choice of (Fedora), they come built in and apparently even slightly modified sometimes to work with the latest build. I ran nvidia-xconfig to get a simple configuration in place, and then restarted gdm.

Instead of Gnome, I was greeted by a white console screen. I logged in and met my first challenge, a relatively easy one:

(==) Log file: "/var/log/Xorg.0.log", Time: Tue Feb 24 02:44:59 2009
(==) Using config file: "/etc/X11/xorg.conf"
Parse error on line 12 of section Files in file /etc/X11/xorg.conf
        "RgbPath" is not a valid keyword in this section.
(EE) Problem parsing the config file
(EE) Error parsing the config file

Fatal server error:
no screens found

The simple fix is to comment out the RgbPath line in the "Files" Section.

A quick 'svcadm restart gdm' presented the next challenge:

(EE) NVIDIA(0): Failed to initialize the GLX module; please check in your X
(EE) NVIDIA(0):     log file that the GLX module has been loaded in your X
(EE) NVIDIA(0):     server, and that the module is the NVIDIA GLX module.  If
(EE) NVIDIA(0):     you continue to encounter problems, Please try
(EE) NVIDIA(0):     reinstalling the NVIDIA driver.
(EE) NVIDIA(0): Failed to initialize the NVIDIA graphics device PCI:1:0:0. 

I pursued this one for a while before I came up with the solution. Apparently the ogl-select "determines at boot time which vendor supplied
OpenGL headers and libraries will be used. The selection of the OpenGL vendor should be automatic and in most cases will not require any configuration." - except it didn't. There is a manual way to set the vendor:

/lib/opengl/ogl_select/nvidia_vendor_select

I restarted gdm again, to find the GLX module now loads nicely, but I still received the same "Failed to initialize the NVIDIA graphics device" message. It seemed like an opportune time to upgrade to the latest Nvidia drivers - 180.29.

The drivers installed flawlessly - and much faster than Linux since drivers don't have to be compiled against a specific kernel. I tempted fate again with a gdm restart, to again fail:

(II) Loading /usr/X11/lib/modules/amd64//libwfb.so
dlopen: ld.so.1: Xorg: fatal: relocation error: file /usr/X11/lib/modules/amd64//libwfb.so: symbol miZeroLineScreenIndex: referenced symbol not found
(EE) Failed to load /usr/X11/lib/modules/amd64//libwfb.so
(II) UnloadModule: "wfb"
(EE) Failed to load module "wfb" (loader failed, 7)
.
.
.
(II) NVIDIA(0): NVIDIA 3D Acceleration Architecture Initialized
(EE) NVIDIA(0): Need libwfb but wfbScreenInit not found

Fatal server error:
AddScreen/ScreenInit failed for driver 0

I suspected that I needed to replace /usr/X11/lib/modules/amd64/libwfb.so and /usr/X11/lib/modules/libwfb.so with the original versions from SUNWxorg-server, as the Nvidia versions appear incompatible with this version of Xorg. Of course, these files were now gone, and of course I had neglected to take a ZFS snapshot.

I decided to try out the 'fix' feature of pkg. I simply ran 'pkg fix SUNWxorg-server' and it correctly identified these two files (and only these two files) as needing replacement. It downloaded them and completed patching in < 30 seconds. My final gdm restart was successful; X started up, and nvidia-settings confirmed the optimized drivers were installed.

Points scored by OpenSolaris over Linux:

* I don't miss Linux's annoying requirement of compiling drivers against specific kernel versions.

* I appreciate that OpenSolaris, like many Linux distros, values end-user experience over open source purity and delivers closed-source Nvidia drivers. I'm certainly tired of performing the Nvidia shuffle on Fedora/RHEL.

* pkg fix is nice - downloading only the delta between what should have been installed and what was saved time.

Points scored by Linux over OpenSolaris:

* There's much more information available on debugging Linux problems than Solaris problems, in sharp contrast to 10+ years ago when I was a full time Solaris junkie.

* WPA2 is flaky. In fact, it may be wireless in general, but on my laptop it randomly flips between trying to use an unplugged wired port and the wireless adapter.

Points scored by OpenSolaris over me:

* Not taking a manual snapshot before installing the Nvidia drivers. ZFS makes it so simple, there's not much of an excuse. OpenSolaris actually takes snapshots for you when you run 'pkg fix' - very nice.