[Beta] 1.3.4 Release

1- First Time enabling pushing stock from Checkout Server side using API
2- Enable or disable manual stock sync
3- Added option to choose what type of the Magento Installation it is (Online store side or CheckoutApp server side), which decides which sync mechanism to use (only stock sync with API is compatible with this so far)
4- Cron Job stock sync is now compatible with Fetch and Push (depending on server type setting in point “3”)

http://www.megafileupload.com/en/file/263519/CheckoutAppSync-1-3-4-tgz.html

[Beta] 1.3.3 Release

Release Update:

1- Sync stock now is with API (fetch on Magento store side) feature
2- Only changed stock are updated, old mechanics of updating every product is now deprecated
3- Password in config is now invisible to user
4- Easier initial setup with one click. (copies metatype values, database name, apply database change to make it compatible with point ‘2’)

http://www.megafileupload.com/en/file/263509/CheckoutAppSync-1-3-3-tgz.html

Stock Sync Code Update

First TODO in the previous TODO list was:
Very heavy server load on product stock sync.
Solution:

• Use the CA “lastmodified” timestamp value “stock” table ALTER introduced in previous post “Starting Over” to compare the changes done since last sync time.
• Create an API that handles the stock update in case stock quantities changes have been detected.

Starting work on this issue, but according to the previous “Fetch Or Push” post, I should make the interface to be on CA’s server side.
Since moving the interface to be on CA’s side will require more work, so I will implement it first normally on Magento’s Online Store side but make it compatible with future interface change.

1- Add “insert“ and “get“ last stock update sync timestamp in preference table (done)
2- Edit current stock query to compare with last stock sync time (done)
3- Add Api Configuration to admin (done)
4- Add Api function that will handle the request, make the query and return result (done)
5- Edit, Add stock sync function to use the Api connection (done)
6- Add cron job to handle auto stock sync using new concept (done)
7- Disable old stock sync concept cron job (done)

Fetch Or Push?

Depending on wether it is to fetch or to push, the work trouble will differ.
I think the advantage of making the sync interface on CA’s server side, will make faster logging in and server will not need a Static IP for the module to work.
Not having a static IP is useful if managing your store from your brick and mortar HQ just needing to push the products online.
Disadvantages are, more custom APIs need to be coded....maybe some more.

Simple TODOs - List 1

Major weaknesses in the module should be handled first:

1- Very heavy server load on product stock sync.
The current mechanism is :

• Magento cron job (10min) checks all products available on its DB.
• Check if they were synced to CheckoutApp(CA) before. If not, it is skipped.
• The confirmed products IDs are used in a SQL query to get all these products current stock in CA.
• Magento spends sometimes over 30sec (the maximum allowed time for the cron job to work) to loop over all the confirmed products to update their stock quantity.

Actually, cron job does not only do this stock sync, some other Magento’s built in functionalities are triggered too, along with any other orders sync in this module....well, cron job is busy.

Solution:

• Use the CA “lastmodified” timestamp value “stock” table ALTER introduced in previous post “Starting Over” to compare the changes done since last sync time.
• Create an API that handles the stock update in case stock quantities changes have been detected.

2- Proper error handling
Catching exceptions in code breaks cronjob (So they were removed). But in manual sync exceptions are caught fine.

3- Proper Tax Sync
Only simple tax rates are implemented currently. No tax groups or locations are implemented in to the sync.

4- Proper Product and Customer info (metavalue and metanumber) Update
Currently, if a certain detail is found different during sync, it is updated. CA actually handles this in a different way. CA inserts the detail as a new record. So when querying a product to show, it queries the last updated detail version.
This issue for customers causes problems. When an order is synced for a certain customer, then later this customer has some details updated, the previous orders will reflect the newest detail and not the detail at the order creation time.

5- Automatic Metatype
Metatype values play the role of mapping the metavalue and metatypes correctly. These values are required in the extension configuration. For convenience, the values input in Magento’s configuration should be done automatically after reading CA’s Metatype table.

Cracking The Code

Checkout’s DB sometimes uses bytea data columns to save some data.
From what I see, some of those are settings data, mostly in the preference table.
Checkout encodes the data before saving it to table, here’s an example:
\\200\\002}q\\001X\\006\\000\\000\\000youngiq\\002J\\315\\215jLs.
The past setting in preference should be holding Enstore’s store name “youngi” and the last sync time.

I found a bytea.py file in checkout’s root which contains few python functions about this encoding and decoding, but I am trying to translate it to PHP in order to use it. (it is a http://python.projects.postgresql.org 2005 project that I can’t find online now)

here’s the content of the file :

'PostgreSQL bytea encodings'

def bchr(ord):
        if not (32 < ord < 126):
                return "\\" + oct(ord).lstrip('0').rjust(3, '0')
        elif ord == 92:
                return r'\\'
        else:
                return chr(ord)

def encode(data):
        return ''.join([bchr(ord(x)) for x in data])

def decode(data):
        diter = iter(data)
        output = []
        next = diter.next
        for x in diter:
                if x == "\\":
                        try:
                                y = next()
                        except StopIteration:
                                raise ValueError, "incomplete backslash sequence"
                        if y == "\\":
                                pass
                        elif y.isdigit():
                                try:
                                        os = ''.join((y, next(), next()))
                                except StopIteration:
                                        raise ValueError, "incomplete backslash sequence"
                                try:
                                        x = chr(int(os, base = 8))
                                except ValueError:
                                        raise ValueError, "invalid bytea octal sequence '%s'" %(os,)
                        else:
                                raise ValueError, "invalid backslash follow '%s'" %(y,)
                output.append(x)
        return ''.join(output)

I tried hard to translate it to PHP, got close but still no success. What I guess is that the input is JSON, so when decoding I expect a JSON output, or at least some nodes and strings.

If anyone has any clue, share it please, coz this is quite important to decode some of the orders info in the DB.

my trial is like follows:

public function bchr($ord)
        {
                if (!((32<$ord)&&($ord<126)))
                {
                        $result = "\\\\". str_pad(ltrim(decoct($ord),0),3,0, STR_PAD_LEFT);
                        Mage::log('1='.$result);
                        return $result;
                }
                elseif ($ord==92)
                {
                        $result = '\\\\';
                        Mage::log('2='.$result);
                        return $result;
                }
                else
                {
                        $result = chr($ord);
                        Mage::log('3='.$result);
                        return $result;
                }
        }
        
        public function encode($data)
        {
                $joined = '';
                $pregData = preg_split('//', $data, -1, PREG_SPLIT_NO_EMPTY);
                foreach ($pregData as $x)
                {
                        $ord = ord($x);
                        Mage::log('x='.$x);
                        Mage::log('ord='.$ord);
                        $joined .= $this->bchr($ord);
                }
                
                return $joined;
        }
        
        public function decode($data)
        {
                $pregData = preg_split('//', $data, -1, PREG_SPLIT_NO_EMPTY);
                $output = '';
                foreach ($pregData as $x)
                {
                        if ($x == '\\\\')
                        {
                                $y = next($pregData);
                                //Mage::log($y);
                                if ($y == '\\\\')
                                {
                                        continue;
                                }
                                elseif (is_int($y))
                                {
                                        $os = $y.next($pregData).next($pregData);
                                        $x = chr(intval($os, 8));
                                        Mage::log($x);
                                }
                                else
                                {
                                        //Mage::throwException('Invalid Backslash Follow $y');
                                }
                        }
                        $output .= $x;
                }
                
                return $output;
        }

Old Vs New Concept

The major drawback in my current module is that all the processing is done from Magento's side. Having this server somewhere and Checkout server in your store will cause major delays. Connections are done from Magento to Checkout's database, processes take place and the final instruction is done.

My thought is to have an API layer on Checkout's server and let it handle all DB connections and give results to Magento.

Since I am aware of Magento's APIs and framework construction, I think I will make the new module actually 2 extensions.

1- Dummy Magento Installation and have a (server-side) extension to handle Checkout DB connection and provide secure API externally.

2- The real Magento Installation to have (worldwide-side) extension.

I think this construction will save me lots of time, coz I will eventually have to kind of (split) the current extension to have a server side, and a worldwide side. So alot of work is already done.

Previously I had in plan to rewrite the Tax sync codes for proper syncing during initial setup, but never had the time to do that. I think this is the main reason people do not succeed getting my extension to work.