Wednesday, December 1, 2010

[Beta] 1.4.1 Release

- Fix typo in Administrate Orders template file
=============
1.4.0 Release Update:
- Modify pushing stock, no need to open API connection if no stock has changed since last sync
- Fix “Administrate Stores” bug for the default database connection.
- Fix bug where push stock for the first time cannot find last sync date
- Added check to auto scheduling of newly placed orders, that if they were scheduled before and still pending, they will not be scheduled again.
- Added support of an extra attribute for products barcode, which if not setup will use SKU instead.
- Avoid SOAP error when pushing stock for big number of products. Stock is split to chunks of 50 products by default or as desired in config.
- Product sync now handles updates in checkout way, by entering new records for metavalues and metanumbers instead of editing found one.
- Customer sync now handles updates in checkout way, by entering new records for metavalues and metanumbers instead of editing found one.
- Administrate store section now has better handling and split processes to several options
- Administrate store “Stock” and “Products” Database Mods now check the database if they have been applied before.

NEW DOWNLOAD LINKS

Rapidshare download:
http://rapidshare.com/files/448895965/CheckoutAppSync-1.4.1.tgz
mediafire:
http://www.mediafire.com/?bn2s8gc2irj2kdn
hotfile:
http://hotfile.com/dl/106215848/2379079/CheckoutAppSync-1.4.1.tgz.html

Saturday, August 28, 2010

[Beta] 1.3.4 Release

1- First Time enabling pushing stock from Checkout Server side using API
2- Enable or disable manual stock sync
3- Added option to choose what type of the Magento Installation it is (Online store side or CheckoutApp server side), which decides which sync mechanism to use (only stock sync with API is compatible with this so far)
4- Cron Job stock sync is now compatible with Fetch and Push (depending on server type setting in point “3”)

http://www.megafileupload.com/en/file/263519/CheckoutAppSync-1-3-4-tgz.html

[Beta] 1.3.3 Release

Release Update:

1- Sync stock now is with API (fetch on Magento store side) feature
2- Only changed stock are updated, old mechanics of updating every product is now deprecated
3- Password in config is now invisible to user
4- Easier initial setup with one click. (copies metatype values, database name, apply database change to make it compatible with point ‘2’)

http://www.megafileupload.com/en/file/263509/CheckoutAppSync-1-3-3-tgz.html

Saturday, August 21, 2010

Stock Sync Code Update

First TODO in the previous TODO list was:
Very heavy server load on product stock sync.
Solution:
  • Use the CA “lastmodified” timestamp value “stock” table ALTER introduced in previous post “Starting Over” to compare the changes done since last sync time.
  • Create an API that handles the stock update in case stock quantities changes have been detected.
Starting work on this issue, but according to the previous “Fetch Or Push” post, I should make the interface to be on CA’s server side.
Since moving the interface to be on CA’s side will require more work, so I will implement it first normally on Magento’s Online Store side but make it compatible with future interface change.

1- Add “insert“ and “get“ last stock update sync timestamp in preference table (done)
2- Edit current stock query to compare with last stock sync time (done)
3- Add Api Configuration to admin (done)
4- Add Api function that will handle the request, make the query and return result (done)
5- Edit, Add stock sync function to use the Api connection (done)
6- Add cron job to handle auto stock sync using new concept (done)
7- Disable old stock sync concept cron job (done)

Fetch Or Push?

Depending on wether it is to fetch or to push, the work trouble will differ.
I think the advantage of making the sync interface on CA’s server side, will make faster logging in and server will not need a Static IP for the module to work.
Not having a static IP is useful if managing your store from your brick and mortar HQ just needing to push the products online.
Disadvantages are, more custom APIs need to be coded....maybe some more.

Simple TODOs - List 1

Major weaknesses in the module should be handled first:

1- Very heavy server load on product stock sync.
The current mechanism is :
  • Magento cron job (10min) checks all products available on its DB.
  • Check if they were synced to CheckoutApp(CA) before. If not, it is skipped.
  • The confirmed products IDs are used in a SQL query to get all these products current stock in CA.
  • Magento spends sometimes over 30sec (the maximum allowed time for the cron job to work) to loop over all the confirmed products to update their stock quantity.
Actually, cron job does not only do this stock sync, some other Magento’s built in functionalities are triggered too, along with any other orders sync in this module....well, cron job is busy.

Solution:
  • Use the CA “lastmodified” timestamp value “stock” table ALTER introduced in previous post “Starting Over” to compare the changes done since last sync time.
  • Create an API that handles the stock update in case stock quantities changes have been detected.
2- Proper error handling
Catching exceptions in code breaks cronjob (So they were removed). But in manual sync exceptions are caught fine.

3- Proper Tax Sync
Only simple tax rates are implemented currently. No tax groups or locations are implemented in to the sync.

4- Proper Product and Customer info (metavalue and metanumber) Update
Currently, if a certain detail is found different during sync, it is updated. CA actually handles this in a different way. CA inserts the detail as a new record. So when querying a product to show, it queries the last updated detail version.
This issue for customers causes problems. When an order is synced for a certain customer, then later this customer has some details updated, the previous orders will reflect the newest detail and not the detail at the order creation time.

5- Automatic Metatype
Metatype values play the role of mapping the metavalue and metatypes correctly. These values are required in the extension configuration. For convenience, the values input in Magento’s configuration should be done automatically after reading CA’s Metatype table.

Thursday, August 19, 2010

Cracking The Code

Checkout’s DB sometimes uses bytea data columns to save some data.
From what I see, some of those are settings data, mostly in the preference table.
Checkout encodes the data before saving it to table, here’s an example:
\\200\\002}q\\001X\\006\\000\\000\\000youngiq\\002J\\315\\215jLs.
The past setting in preference should be holding Enstore’s store name “youngi” and the last sync time.

I found a bytea.py file in checkout’s root which contains few python functions about this encoding and decoding, but I am trying to translate it to PHP in order to use it. (it is a http://python.projects.postgresql.org 2005 project that I can’t find online now)

here’s the content of the file :

'PostgreSQL bytea encodings'

def bchr(ord):
        if not (32 < ord < 126):
                return "\\" + oct(ord).lstrip('0').rjust(3, '0')
        elif ord == 92:
                return r'\\'
        else:
                return chr(ord)

def encode(data):
        return ''.join([bchr(ord(x)) for x in data])

def decode(data):
        diter = iter(data)
        output = []
        next = diter.next
        for x in diter:
                if x == "\\":
                        try:
                                y = next()
                        except StopIteration:
                                raise ValueError, "incomplete backslash sequence"
                        if y == "\\":
                                pass
                        elif y.isdigit():
                                try:
                                        os = ''.join((y, next(), next()))
                                except StopIteration:
                                        raise ValueError, "incomplete backslash sequence"
                                try:
                                        x = chr(int(os, base = 8))
                                except ValueError:
                                        raise ValueError, "invalid bytea octal sequence '%s'" %(os,)
                        else:
                                raise ValueError, "invalid backslash follow '%s'" %(y,)
                output.append(x)
        return ''.join(output)

I tried hard to translate it to PHP, got close but still no success. What I guess is that the input is JSON, so when decoding I expect a JSON output, or at least some nodes and strings.

If anyone has any clue, share it please, coz this is quite important to decode some of the orders info in the DB.

my trial is like follows:

public function bchr($ord)
        {
                if (!((32<$ord)&&($ord<126)))
                {
                        $result = "\\\\". str_pad(ltrim(decoct($ord),0),3,0, STR_PAD_LEFT);
                        Mage::log('1='.$result);
                        return $result;
                }
                elseif ($ord==92)
                {
                        $result = '\\\\';
                        Mage::log('2='.$result);
                        return $result;
                }
                else
                {
                        $result = chr($ord);
                        Mage::log('3='.$result);
                        return $result;
                }
        }
        
        public function encode($data)
        {
                $joined = '';
                $pregData = preg_split('//', $data, -1, PREG_SPLIT_NO_EMPTY);
                foreach ($pregData as $x)
                {
                        $ord = ord($x);
                        Mage::log('x='.$x);
                        Mage::log('ord='.$ord);
                        $joined .= $this->bchr($ord);
                }
                
                return $joined;
        }
        
        public function decode($data)
        {
                $pregData = preg_split('//', $data, -1, PREG_SPLIT_NO_EMPTY);
                $output = '';
                foreach ($pregData as $x)
                {
                        if ($x == '\\\\')
                        {
                                $y = next($pregData);
                                //Mage::log($y);
                                if ($y == '\\\\')
                                {
                                        continue;
                                }
                                elseif (is_int($y))
                                {
                                        $os = $y.next($pregData).next($pregData);
                                        $x = chr(intval($os, 8));
                                        Mage::log($x);
                                }
                                else
                                {
                                        //Mage::throwException('Invalid Backslash Follow $y');
                                }
                        }
                        $output .= $x;
                }
                
                return $output;
        }

Wednesday, August 18, 2010

Old Vs New Concept

The major drawback in my current module is that all the processing is done from Magento's side. Having this server somewhere and Checkout server in your store will cause major delays. Connections are done from Magento to Checkout's database, processes take place and the final instruction is done.

My thought is to have an API layer on Checkout's server and let it handle all DB connections and give results to Magento.

Since I am aware of Magento's APIs and framework construction, I think I will make the new module actually 2 extensions.

1- Dummy Magento Installation and have a (server-side) extension to handle Checkout DB connection and provide secure API externally.

2- The real Magento Installation to have (worldwide-side) extension.

I think this construction will save me lots of time, coz I will eventually have to kind of (split) the current extension to have a server side, and a worldwide side. So alot of work is already done.

Previously I had in plan to rewrite the Tax sync codes for proper syncing during initial setup, but never had the time to do that. I think this is the main reason people do not succeed getting my extension to work.

Starting Over

It's been a while we had our store working with the magento-checkout extension I wrote. But obviously, it's buggy. I figure out how to avoid all bugs coz I wrote the code, but I want to share everything and so everyone can benefit from my work.

So, I am rewriting the whole thing. I will revamp the whole extension.
I have been testing with Checkout's integrated ecommerce (EnStore). First it will not work in my country, nor with my currency. So to test, I put location the US, and USDollar currency.
Also Enstore is trying to lease the hosting, which makes it inconvenient of using a "username.enstore.com" address instead of your own address (like "blabla.com").

From what I see, they are making Enstore work as a interface to Checkout's DB, they will provide APIs for integration. Also it is very under construction and far from being done, this is why it is for free while in beta. Magento has lots of payment and shipping methods too, and very integratable, and since we have access to Checkout's DB then we can work on a clean Sync solution.

I will put in mind to be able to Sync in a Checkout->Magento way. My previous module was a Magento->Checkout solution, maybe I will use some features in the end to enable sync both ways.

First thing I noticed about integrating Enstore with Checkout. Preferences table holds the webstore identification and last sync time.
The sync was meant to be MANUAL, I will put in mind to make possible syncing to Magento Automatically.

Products info (like name, price,barcode...etc) all are saved in metavalue and metanumber tables along with a timestamp to figure out if this was changed after last sync was done... (that is very helpful).

BUT, STOCK....stock does not have a timestamp in stock table. Obviously in Enstore, there are no quantities being added during sync. This is not convenient as what if the product is out of stock, you shouldn't have to manually disable the product from being sold, this should be done automatically.

I figured out how to add a timestamp column to the stock table that auto updates on creation or on modification.(thanks to POINTBEING.NET). So I will modify the stock table, which I think will make no problem upgrading later to newer versions of Checkout...but hey let's see over time.

The code for adding this column is:

ALTER TABLE stock
ADD lastmodified TIMESTAMP;

ALTER TABLE stock
ALTER COLUMN lastmodified
SET DEFAULT CURRENT_TIMESTAMP;

UPDATE stock
SET lastmodified=CURRENT_TIMESTAMP;

CREATE OR REPLACE FUNCTION update_lastmodified_column()
RETURNS TRIGGER AS '
BEGIN
NEW.lastmodified = NOW();
RETURN NEW;
END;
' LANGUAGE 'plpgsql';

CREATE TRIGGER update_lastmodified_modtime BEFORE UPDATE
ON stock FOR EACH ROW EXECUTE PROCEDURE
update_lastmodified_column();


Keep checking for news

Saturday, June 12, 2010

Some Basics

I had few emails asking some basic questions, and to make usefulness to everyone I'm posting my replies here along with the latest versions.

Well first of all, The HOST STATIC IP in the server configuration should not be "localhost".
Your magento installation is at your remote server(most probably) while your checkout installation is on your computer, what you're telling magento is to try to connect to the postgresql installed on the same machine as magento which is incorrect.

I highly suggest you try everything on your local machine.
A few tips on getting this host locally:

Make a backup of your magento store from System->Tools->Backups.
download it from your var/backups on your magento server.
Then compress your installation files to a zip file (you can skip the media folder coz it must be big and full of images), you can skip var/backups because it will be full of your database backups, and you can skip the downloader folder too.
Then host this magento installation on your computer (use MAMP, it's great).
use "Sequel Pro" to connect to magento's mysql database and make sure that you change core_config_data table the "base url" and "base secure url" to "localhost". if you don't do this, magento will not display properly on your browser because it will not be able to find its required files (css and images)

This way you can test further and be sure you will not break your real shop.

One thing to clear out. At the end, when you want to make the true connection with your live installation and your checkout installation, you must make sure the machine where checkout is installed does have a STATIC IP. This is a must so that your remote server will be able to find it and connect to it each time it needs. Or else you will not be able to work live.
Probably if you do not have STATIC IP, your ISP will charge your for it a little but not much.

The postgresql database you should be connecting to should be Checkout App's. And if everything's set to default when you first created Checkout App's DB then the port should be 5505. Also if Checkout is installed on your computer and your magento is installed on the same one then the host should be "localhost" or "127.0.0.1" depending on if connecting through socket or not.

Going to Administrate stores was meant to show you Checkout App's DB name so you put it in the configuration of the extension, but I was not in focus that time that I wrote the code to use the DB name set in the configuration...so to bypass this issue, put in the configuration DB name: postgres.

After you get the Administrate stores connecting to your server and retrieving your Checkout DB name, put it back in the config.
Administrate stores will also help you set your store order number.

If you go through this and you face other issues, send me back, i'll do my best to help you with your project.

Version 1.3.2Beta
http://www.megafileupload.com/en/file/240535/CheckoutAppSync-1-3-2-tgz.html

Friday, April 23, 2010

Beta 1.0.0 Release

Just done packaging newest Release.
Release submitted to Magento Connect and awaiting approval.

Fixed issues with installation and database tables creation.

Features:
-Customers sync/detect changes and update
-Tax rates sync
-Sync orders
-Sku, name, prices(and special price), cost, tax assigned for simple products
-Sync stock to checkoutapp (auto create a purchase order like checkoutapp's own products import from file feature)
-Sync variations to checkoutapp (product attributes in magento)
-Sync stock back to Magento
-A Sync Schedule for auto order push and auto import stock to update magento's
-Cron Jobs for automation(depends on the sync schedule)
-Update Order Info (print packingslip, shipping rate calculation)
-Aramex Shipping AirWayBill PDF creation

Download link: http://www.megafileupload.com/en/file/221551/MageCheckoutAppSync-1-0-0-tgz.html

Saturday, March 27, 2010

Progress

Here's the progress so far:

-Customers sync/detect changes and update (done)
-Tax rates sync (done)
-Sync orders (done)
-Sku, name, prices, cost, tax assigned for simple products(done)
-Sync stock to checkoutapp (done...by auto create a purchase order like checkoutapp's own products import from file feature)
-Sync variations to checkoutapp (product attributes in magento) (done)
-Sync stock back to Magento (done)
-Automate these actions somehow, like with cron jobs or trigger after event (done...only implemented for auto order push and auto import stock to update magento's)
-Create a kind of log of each process/sync done for revising and error handling/fixing situations (done...only for the automated features, logs errors and success)

I'm planning to release a beta version asap, and a demo video.

Wednesday, March 3, 2010

Where am I - Roadmap

One thing I want to clear:
All my work is goin in the way of "syncing Magento TO Checkout App"...meaning my codes will not be as useful for those who already work on Checkout as for those who work on Magento and just want to add POS like me.

Concept is, sync tax, customers, products, stock and orders.
syncing orders will run the customers sync codes along the process to successfully assign the customer to the synced product with updated shipping addresses.
I already implemented the tax sync, but for me, in Egypt, we have only sales tax of 10%, no tax regions or any other special or compound rates...So I only implemented syncing the tax rate as a tax group in Checkout.

I am currently in the middle of coding the simple products sync.
The plan is to successfully implement :
-Customers sync/detect changes and update (done)
-Tax rates sync (done)
-Sync orders (partially done-without products or amounts)
-Sku, name, prices, cost, tax assigned for simple products(in progress)
-Sync stock to update inventory total costs and ledger
-Use of variations to sync configurable products and their associated simple products
-Sync stock back to Magento
-Automate these actions somehow, like with cron jobs or trigger after event
-Create a kind of log of each process/sync done for revising and error handling/fixing situations

I really am not finding hard time coding, and I really encourage people to start working with me on this module because in 10 days I am 30% done of what I need, so this is promising.
I also want to clear out that since I'm not just developing this for the community but first of all solve my work case. Then, not all features can be implemented in this module by me since I don't have the time, and I don't have the case to work on.

Maybe we can move this project on to bazaar or other similar project management solutions

Tuesday, March 2, 2010

Alpha 0.2.0 Release

New in this version:
-Real Magento Order Number is added to the order as a note. This makes searching for orders with Magento's # really easier from the orders list search bar.
-Sync Tax Rates from Magento to Checkout App. This is for simple tax rates, no tax regions.

Note: Complete Tax sync can be coded easily to implement all Magento's power, but for my project I am satisfied with syncing the simple tax rates. Feel free to improve the tax codings to adapt to your needs

Download Alpha 0.2.0

Monday, March 1, 2010

Alpha 0.1.0-Release

So far what I reached is complete customer sync to Checkout from orders.
You will be able to mass select orders, choose "Sync Orders" from the mass options, customer is then created/updated(update concept tested and is OK). Shipping addresses are checked and if the order's shipping address is not found it is created and assigned to the synced order.

No items, amounts or totals sync codes are written yet.

This release is just for curious people who want to really know if this project is doing fine :)
Comments and suggestions are very welcome.
enjoy :)
Download Alpha 0.1.0

Accessing CheckoutApp Database

At first, be sure you firewall your network to checkout's server or else you might have security risks. It is recommended to only forward TCP port 5505 on which Checkout's Postgres server is accessed and to enable access of WAN to only your Magento's host IP for extra security. In order for this Magento module to work, your Magento host server will have to be able to reach Checkout's server (normally this means to have server set on a static IP)

Locally, if you want to access Checkout's DB, first launch Checkout. The welcome window will show, and a loading wheel on the top right will indicate server is being started. Once server started open your preferred GUI Postgres Admin (like pgAdmin or Navicat).



When you setup a new store in Checkout, a 6 character Database Name is used, and non-related to the name you have set.
Checkout comes with a preset demo store...but let's just try to look at which databases are available on the server first.

Set in pgAdmin host as localhost (or any other adequate IP).
set port: 5505
username: admin
password: admin
set maintenance DB the default postgres

you should now be able to login.



Take note of the required DB name, we will need to input this in the module's configuration in Magento. (note that if you did not setup a new store, you should be able to see a DB already there which is the demo store)

Where to get it


CheckoutApp's crack that was released just resets the demo time period. Every 30 days you will have to overwrite a file to get access back to your store.
http://www.mac-bb.org (a really cool warez forum) has the application as well as the crack...little work for you to do here

This time reset is done with 1 file called: com.madebysofa.Checkout.plist
you have to replace it over at ~YOUR HOME/Library/Preferences
so far, you will be able to login to your store but you will see the count of days left reset to 30 days.
So to override this you replace English.lproj folder over at checkoutApp...to do so, go to your Applications folder, right click on Checkout and click "Show Package Contents" and replace the English.lproj in Contents/Resources/English.lproj

now, you will see that the copy of checkoutApp is licensed.

Saturday, February 27, 2010

[Intro] First Magento POS sync Module-Checkout App Mac POS


hey everyone,

I have been the creator/maintainer of an online skate shop for more than a year now.
From the first moment I chose Magento to be my project foundation I knew I took the right decision...Modularity of magento along with very wide availability of modules as reference and guiding documentation helped me alot to improve my php programming.

At first, I was very noob in php, only knew the very basics to just customize the small requirements of the shop. And I always checked that the admin modules are adequate for daily work flow.

For 4 months now I've been researching the sync of Magento with a POS for our walk-in customers as well as for invoicing flexibility...Magento is powerful for online/phone orders but not helpful having 10 customers waiting for 15min to get their invoices, pay and leave.

Well, I'm a very big fan of Mac, and I believe it's a very powerful OS to work on, especially that security risks and viruses are very rare. So I searched for solutions to not depend on OS first so that we don't have to throw all the PCs we have in our office.

First came OpenERP magento module, worked on it for 2 months but honestly it's a big project and it's far from being ready to implement...yet I believe when it's done, it's gonna be huge success. OpenERP is powerful ERP system, but customization needs someone advanced in python, and since all my knowledge is html/php/js I tried to stick to these because I don't even have the time to learn new stuff like python.
Checked Adempiere's POS, posterita...lack of features and poor interface....why reinvent the wheel.

Tried Quickbooks POS, it has plenty of features, but the module of integration with Magento from eCC webgility costs a fortune, especially if you need inventory sync...so this was out of the way. Few other Quickbooks POS integration modules are there, but they are expensive, plus I then remembered I don't want to stick to one OS, especillay to be Windows, it has always been a pain having to format the PCs from time to time to fix them.
Quickbooks also lacks the power of having an SQL database, as well as remote access...Quickbooks is amazingly powerful for single terminal single stores, highly recommended, but in case of online orders, the integration is not easy to achieve.

Since I'm a big fan of Mac, I've searched for the available POS, and came to realize that Checkout App POS is fantastic, I'm sure alot of people will say "hey, lightspeed is way amazing", I'd say "Filemaker database not that simple to access and work on as much as PostgreSQL on which is App is based. This makes it alot easier to integrate and hey I'm doing this project as Open Source, so lightspeed hasn't been cracked so far....Checkout App, oh yeah it has been [K]'ed"

well, enough introduction. Since I believe this is the first Open Source Magento POS solution module project, I thought of blogging my achievements for knowledge and learning. Feel free to contribute, and I will start blogging all I've done in the past week in this project...achieved quite beautiful results.
CIAO