Gamling.org was DDOSed

And this is my first experience with DDoS attacks.
It was funny how the server is blazingly fast after I enabled APF.

Therefore, of course, Gamling.org was nuked with DDoS.


What I did is that I use Advanced Policy-based Firewall (APF) and configure to allow only specified ports.

DEVEL_MODE="0"
IG_TCP_CPORTS="21,22,25,53,80,110,143,443,3306"
IG_UDP_CPORTS="53,111"
USE_AD="1"

And I have installed Brute Force Detection (BFD) and DDoS Deflate in order to detect who nukes Gamling.org.


And I also add RootKit Hunter to check and remove Spyware from the server. (I didn't know that Linux also has Spyware :S)

ActionMailer Testing with Rspec

In environment/test.rb

Flood::Application.configure do
   config.action_mailer.delivery_method = :test
end

And in your Rspec case, after sending an email, you can validate it with ActionMailer::Base.deliveries, which is an array of emails sent.

For example:

ActionMailer::Base.deliveries.last.to.should =~ ["test@gmail.com"]

Scaling Gamling.org

TBD

Gamling.org bottleneck

I have found that the bottleneck for Gamling.org is on Mongodb.

I think there is a problem with Passenger forking. I have to try its solution first.

Right now I'm figuring out how to optimize it.

Here is the CPU usage:

Linux Server optimization cheat sheets

- netdev's backlog
- somaxconn
- open file limit

Nginx 502 Bad Gateway

It occurs somewhat often on Gamling.org. I'm still trying to figure out what really happens.

I have found on the log and this is the error message:

unix:/passenger_helper_server failed (11: Resource temporarily unavailable) while connecting to upstream

The solutions I have tried

1.
One proposed solution is to increase the buffer size for header.
(http://stackoverflow.com/questions/2307231/how-to-avoid-nginx-upstream-sent-too-big-header-errors)

It does not seem to solve the problem.

2.
The second solution is to disable firewall. It does not solve the problem.


3.
This is the one that actually solves the problem.

The problem is that the backlog queue (for imcoming connection) is full. Therefore, we shall increase it.

The default value of net.core.somaxconn is 128.

I have increased it to 40,000! with this command, sysctl -w net.core.somaxconn=40000.

The default value of net.core.netdev_max_backlog is 1000. I have increased it to 40,000 with this command sysctl -w net.core.netdev_max_backlog=40000.

Also do not forget to set open file limits:

just increase it to 100,000 with ulimit -n 100000

And edit this file /etc/security/limits.conf


That does seem to solve problems just a little.

The real problem is that when Passenger spawns a new instance, it clones a previous process. It also clones THE SOCKET FILE.

This means that all instances use the same socket to connect to MongoDB and Redis.
And there you go a bottleneck!

Remote SSH the right way

http://articles.slicehost.com/2009/5/13/capistrano-series-setting-up-git

Beware of overriding a controller

The problem that I have found is that

I override one method of a controller in RSpec, and it totally messes up the routing...

Match route in Rails 3 cannot parse a comma

Never store a HTML element in a variable

Store its ID instead.

Because if its HTML source is moved or replaced, the previous element is destroyed and the new element is created with the same ID.

Production to serve static asset on mounted rack app

use config.serve_static_assets = true

Clearing cookies on Selenium

You will need to go to the specified domain in order to clear cookies related to that domain

Selenium Superbug (Clicking problem)

Today I have just found out that Selenium WebDriver won't click on anything that is not on the current view.

This problem occurs because my HTML is somehow strangely designed that it makes Firefox(and Selenium) miscalculates the position of a target button. (My guess is floating css causes it.)

Selenium always scroll to the button before the clicking happens.

This time the button is not even in Firefox's view.

The direct fix would be to redesign the HTML or fix Selenium or Firefox.

Well, I cannot do that. So what I have done is changing the stylesheet of the button before clicking in order to make it visible in the view, and revert it back later.

The css I have added is {position:fixed; left:0px; top:0px;}

Ajax on Rails 3's session does not work

It is something about CSRF meta tag. Please take a look:

Here

and

Stackoverflow's answer

Lesson learned: never use kind_of?

Always use respond_to?

I have found the problem when doing the integration test and invoke the function to process the notification queue.

Somehow, there are 2 processes doing this. So, the same class in a source file might refer to different classes (identical name, but different object_id).

This presents a problem to kind_of? because it will return false, even the inheritance is correct.

Strange enough, kind_of? is only wrong when you use it against its superclass. Therefore, I think instance_of? won't have this problem.

A procedure to migrate database

1. Backup the database file into .bak
2. Make a change on CayenneModeler, save the XML
3. Generate migration SQL
4. Start Java application (because the migration will be run automatically) OR run the migration explicitly.

Database-related technology for Java application

One of the crucial things to develop a system is to maintain the version of the database schema.

Rails has the built-in procedure, which makes the development painless.

I have been looking into an ORM framework together with a database migration framework.

H2, Cayenne with Flyway are chosen.

H2 is chosen because it supports standard SQL and it is embeddable, yet still a powerful database.

H2 can be run with a mixed mode inside your Java application. The mixed mode means that your Java application can access H2 natively, while other applications (e.g. Rails) can access H2 through TCP.

And it performs well with many concurrent connections. (This is a big win over SQLite.)

Flyway is chosen (over Liquibase) because it offers Java APIs. This means that I can run migration at every startup of the server. You can also run migration from command-line, while Liquibase offers only command-line tools.

The good thing is that Liquibase offers portable migration script because it uses XML instead of raw SQL. However, we do not switch database that often.

Cayenne is chosen over TopLink and Hibernate because of its simplicity. Cayenne offers a GUI tool to manipulate the schema and generate SQL migration (that can be saved and used by Flyway.

TopLink and Hibernate are too good and, of course, more complicated.

Performance is not considered here because there are not many clients (50 at most). Therefore, any normal ORM will be ok.

-----

I have just found one con. The Cayenne GUI tool cannot create indexes...

The good way to generate secure random key

Selenium 2.0 dark secret has been unlocked

Never ever focus on the Firefox window, which is being tested with Selenium.

This is because it will mess up the events and some clicking, specifically <a> and <button>,will fail.

(I have done a solid experiment on it. I have tested 4 elements, span, anchor, button, and input[button]. Only anchor and button have the problem)

----------------

OMG, I have solved it.

By focusing and send_keys "\n", the clicking problem is gone !!

Please be aware that the clicking problem only occurs with the anchor tag, and this approach does not work with other tags.

Therefore, please use the below code:



YESSSSS!


--------------------

I have found a problem with <button> that contains a span inside. But I could not repeat the problem. So, I'll just make a note right here for future reference.

Watir on Rails

I have been using Selenium + Capybara for a week now. Somehow, the clicking on Selenium is super unstable. The test cases occasionally fail.

It is really strange because Watir also uses selenium webdriver. But anyway the problem has gone now.

I have decided to give Watir a try. Anyway, Watir does not have an easy way to integrate with Rails 3.

Therefore, I have decided to build a gem for that.

And I proudly present you my first gem:

watir-webdriver-rails

I'm so friggin' happy :D :D :D

The gem is hosted on Rubygems.org. Therefore, you can install gem through the normal way

Selenium sometimes does not click on anchor tag with onclick

I don't know why it does not perform the clicking.

This upsets me so much. I thought Selenium is stable.

Now I'm moving to watir-webdriver, which partially use selenium-webdriver. But it does not seem to have any problems.

According to a StackOverflow's answer, we should use fire_event('click') instead.

But Capybara does not support fire_event()..

---------------

I have tried fire_event() with Watir.

Here is the result:

It works with any element with the onclick attribute. But it does not work with <a href="http://www.google.com">.

---------------

I am getting to the bottom of this problem now.

If the link we click has no CSS class, then everything is ok even we switch to that Firefox and move mouse over it.

If the link has a CSS class, but the class has no :hover and no :active, then everything is still ok.

If the link has a CSS class, but the class has :hover but no :active, then everything is still ok.

If the link has a CSS class, but the class has :active but no :hover, then Selenium will fail to click on an anchor if you move mouse over Firefox being tested.

It is so strange....

Architectures for a multi-client system

The first architecture design for a multi-client system is to:
- Open 2 threads for a single connection to client (One for reading, and another for writing)
- When a job come in, we fork a thread to do some jobs.

The problem with this architecture is that if there are 2 jobs that should not be done concurrently, then there is a big problem.

We can use the synchronized statement, but the synchronized statement cannot guarantee the order of jobs.

An example from POS system that breaks this architecture is when a client opens a table and the very same client make orders immediately after.

There is a chance that the make-order job will be processed before the open-table job.

----------------

The second design is:
- Open a single thread for a single connection. Therefore, this implies that when a request is sent, it expects a result. (Because there is only one thread for read and write.
- We use that same thread to process a job (since it has to give back the response anyway.)

This one is the current architecture. The jobs are nicely put in order.

But it loses performance in some cases.

The example that breaks it is when I open table 5 and make orders on it, then I open table 6.

It turns out that the open-table-6 request has to wait because the requests must be executed in order.

And there is a performance penalty when pushing an update. Because it has to wait until there is no request to be processed, then the thread will be available for writing update to the client.

-------------------

The third architecture is superior to those two.

- There are 2 threads to read and write socket. The thread of reading socket also translate request and add job to the queue.
- Queue is divided into multiple queue. Ideally, one queue for one table. This way we can ensure the order of execution.
- A job is dequeued by a worker. When a worker finishes, it posts a response to the writing thread.

With this design, everything is asynchronous. This means that each message must be labelled with a number. Otherwise, when we post a response, we won't know which request the response belongs to.

The implementation is a little bit harder though, but the abstractions are the same. We just process jobs differently. Therefore, it is just difficult on the architecture level, not the code itself.

I think this justifies the third architecture. We cannot just discard this architecture because it is difficult to implement...

Benchmark on the synchronized statement - Java

Here is the result:



It is somewhat strange that ratio for 100,000 loops is bigger than for 1,000,000 loops. Maybe JVM optimizes something for us.

Now if you are going to design a system with the synchronized statement, please beware of the performance detriment.

If you do not call the synchronized more than 10,000 times (within a minute), I would just use the statement and go with a simpler architecture design.

Groovy vs. JRuby

From what I have been researching, there is only one crucial difference between the 2 languages.

JRuby imposes no type on variables. This makes it extremely difficult to use Java's library because you will have an headache with overloaded methods and classes with generic types.

Groovy has type as an option. You can choose to have a type or not. With this, you can easily use Java's library.

And another thing to mention is that if you dream about using JRuby on Rails, then you should stop now. Many gems are written in C, which is impossible to use them with JRuby.

So think carefully before using JRuby because you would have to reinvent many wheels.

If you edit spec_helper.rb, please restart Spork

Otherwise all your test cases will have the error, StackOverflow.

Capybara and the problem with window.opener

I have encountered a really strange problem. When Capybara opens a new window with <a href=".." target="..">.

The new window cannot call window.opener in its Javascript code.

The problem occurs because Capybara starts the first browser with http://127.0.0.1:57214. But my <a href> points to http://localhost:57214.

They points to different domain names. That's why Firefox does not allow reference between windows.

It is amazing how the problem is not really relevant to Capybara.

Anyway, the way to solve it is to change default domain name of Capybara.



Put the above lines somewhere in spec_helper.rb

Capybara not to turn into a blank page after testing

It's a little bit of hacking. You have to go to this file.

C:\Ruby192\lib\ruby\gems\1.9.1\gems\capybara-1.0.0\lib\capybara\selenium\driver.rb

in function reset!, please comment the line @browser.navigate_to('about:blank')

-----

And sometimes when running tests without Spork, Capybara automatically closes browser after the testing is done.

If you want to avoid that, just comment the lines in the same file. The line that calls "quit"

linecache19 errors on missing vm_core.h (Windows 7)

The problem of missing vm_core.h. It should reside in %RUBY_PATH%\include\ruby-1.9.1\ruby-1.9.2-p180.

The cause is that you didn't install Visual Studio C++. Therefore, the C source code for Ruby is not installed as well.

Just install Visual Studio C++. And everything will be fine.

Test GitHub:gist

JRuby cannot find bundle binary file when do bundle exec

When I run:



It tells me that it cannot find command because the spork gem is not installed. But I already installed it under vendor/gems.

To bypass that, please do:


This will install binaries of gems under RAILS_ROOT/bin.
And you can run:

Batch file that forward arguments

You can use %* for all arguments.

Or refer to %1 %2 ... %9 for the first 9 arguments.

Clear cache of faster_require gem on Windows

faster_require

On Windows, we cannot run Gem binary file. Therefore, we cannot clear cache as it is told in the instruction.

Nevertheless, faster_require keeps cache in /your_home_dir/.ruby_faster_require_cache

Just delete it.

Install gem from GitHub

Download the package first. It should contain .gemspec file.

Then do this:

gem build GEMNAME.gemspec

gem install gemname-version.gem

Capybara + RSpec to test Facebook connect

In order to set up Capybara with Facebook connect, we need to set up a few things:

1. You need to specify a server port for Capybara and set localhost:port in your Facebook App setting. Otherwise Facebook won't redirect user back to your URL (security reason)

In order to do that, just drop this line in spec_helper.rb:

   Capybara.server_port = 57124

2. You need to properly handle Facebook login and allow page. Here is the method that I use:

  def facebook_connect(username,password)

    

    if current_path == '/login.php'

      within_window "Log In | Facebook" do

        fill_in 'email', :with => username

        fill_in 'pass', :with => password

        click_button "Log In"

      end

    end

    

    if current_path == '/connect/uiserver.php'

      click_button "อนุญาต" rescue ""

      click_button "Allow" rescue ""

    end

  end

That's it.

Invalid multibyte char (US-ASCII) with Ruby

This is dark magic. The trick to fix is to put this line at the beginning of every file.

# encoding: utf-8

Mongoid Criteria does not work as expected

You have to really be careful because the results are not yet pull

For example, if you invoke .length, it will instead return the length of the criteria array.

Therefore, please always do this:

  result = Kratoo.only(:title).skip(100).entries

skip() in MongoDB is not good

We should design the UI in a way that it uses Range-based paging.

For example, view by date.

Capybara in the house

Just include the gem 'capybara' in your Gemfile.

Then drop these lines in spec_helper.rb

  require 'rspec/rails' # find this line



  require 'capybara/rspec'

  require 'capybara/rails'

  

  Capybara.default_driver = :selenium

And it works like a charm

Stub Time.now with RSpec

Time.stub!(:now).and_return(Time.now + 86400)

Ruby super metaprogramming

I'm posting as a reference. What it does is that it will turn every method in a class into a class method and all of them will be one-level curriable.

class Validator

  

  @@__being_checked_method = nil



  def self.method_added(method_name)

    

    puts "#{method_name} added to #{self}"

    

    return if method_name == :validate

    return if method_name.to_s.match(/^validation__proxy__/)

    return if @@__being_checked_method == method_name

    

    @@__being_checked_method = method_name

    

    puts method_name

    new_name = "validation__proxy__#{method_name}".to_sym

    

    alias_method new_name, method_name

    

    define_singleton_method(method_name) { |*args| 

        puts args.length

        puts args.inspect

    

        l = lambda(&(self.new.method(new_name))).curry

        

        args.each { |a|

          l = l.curry[a]

        }

        

        return l

    }



  end



  class << self

    def validate(lamb_func)

      puts "hello"

    end

  end

  

end



class KratooValidator < Validator

  

  def max_length(length,value)

    puts "#{length} #{value}"

  end

  

  validate max_length(5)



end





puts KratooValidator.new.validation__proxy__max_length(5,10)

puts KratooValidator.max_length(5).call(12)

embeds_many is not an Array

You cannot delete element with a usual mean. You have to use destroy_all

Mongoid cannot have same type of embeds_many

If you have multiple embeds_many which are of the same class, every array you save will be on the first field.

  embeds_many :agree_names, :as => :notificator, :class_name => "BaseNotificator"

  embeds_many :disagree_names, :as => :notificator, :class_name => "BaseNotificator"

In order to prevent this, just change :as to be different. And do not forget to change embedded_in.

  embeds_many :agree_names, :as => :agree_notificator, :class_name => "BaseNotificator"

  embeds_many :disagree_names, :as => :disagree_notificator, :class_name => "BaseNotificator"

And in the embedded class, do this:

  embedded_in :agree_notificator, :polymorphic => true

  embedded_in :disagree_notificator, :polymorphic => true

Wait until MongoDB write the last change

We use getLastError() to wait for it (only the current connection).

But if you would like to wait for the next commit across all connections, you would have to add :fsync=>true.

Enforce uniqueness in a field and discard nil value

We use sparse option on MongoDB.

Moving from DelayedJob to Resque

In order to run Resque on Windows, we require a little bit of tweak in command.

This is to run resque's workers:

   rake environment resque:work QUEUE="*"

This is to run montor website:

   bundle exec resque-web -F

* you need -F option

Delayed Job with RSpec

In your test code, just use this line to execute delayed job:

Delayed::Worker.new.work_off

Rails 3 Madness

I have been struggling with Rails 3 testing framework for a few days now.

This is a madness. There are so many gems to install:

gem 'rspec', :group => [:development, :test]

gem 'rspec-rails', :group => [:development, :test]

gem 'database_cleaner', :group => :test

gem 'factory_girl_rails', :group => :test

gem 'mongoid-rspec', :group => :test



gem 'cucumber-rails', :group => :test

gem 'capybara', :group => :test



gem 'win32-process', :group => :test

gem 'spork', :git => "https://github.com/timcharper/spork.git", :group => :test

rspec-rails is obvious. It is RSpec for Rails.
database_cleaner is for truncate database. It supports Mongoid. That's why I use it.
factory_girl_rails is a mocking library for Rails. It's so simple.
mongoid-rspec is an adapter of RSpec to Mongoid.
cucumber-rails is a BDD testing framework for Rails.
capybara is for testing AJAX and javascript. It can fire an event click on a button or check if the result HTML contains some specific values.
Spork preloads the Rails environment, so that the testing runs faster.

Now you'll need to install RSpec

rails generate rspec:install

Before you run spork, you'll have to make some changes to application.rb in order to make Spork automatically reload any changes

  class Application < Rails::Application

   .

   .

   .



   if Rails.env.test?

      initializer :after => :initialize_dependency_mechanism do

        ActiveSupport::Dependencies.mechanism = :load

      end

    end 



  end

And in spec_helper.rb, we put this line at the bottom of Spork.prefork block:

    ActiveSupport::Dependencies.clear 

And in Spork.each_run block, we put in these lines:

  load "#{Rails.root}/config/routes.rb"

  Dir["#{Rails.root}/app/**/*.rb"].each { |f| load f } 

Now you can start Spork

bundle exec spork --bootstrap

bundle exec spork

Related links: https://github.com/RailsApps/rails3-mongoid-devise/wiki/Tutorial

Eccentric Mongoid behaviour

If you field's suffix is _id or _ids, when you make a query, it will turns your querying string into BSON::ObjectId automatically.

Therefore, please beware.

Add custom field to sunspot

In schema.xml, we add certain tags:

<types>
<fieldType name="autosuggest" class="solr.TextField" positionIncrementGap="100">
   <analyzer type="index">
     <tokenizer class="solr.LetterTokenizerFactory"/>
     <filter class="solr.LowerCaseFilterFactory"/>
     <filter class="solr.NGramFilterFactory" minGramSize="1" maxGramSize="25" />
  <filter class="solr.ThaiWordFilterFactory"/>
   </analyzer>
   <analyzer type="query">
     <tokenizer class="solr.LetterTokenizerFactory"/>
     <filter class="solr.LowerCaseFilterFactory"/>
   </analyzer>
 </fieldType>
</types>

<fields>
<dynamicField name="*_as" type="autocomplete" indexed="true" multiValued="true" stored="false"/>
<dynamicField name="*_ass" type="autocomplete" indexed="true" multiValued="true" stored="true"/>
</fields>

Please note that *_ac is for non-stored field and *_acs is for stored fields. It's just how Sunspot stores fields and we have to follow the convention.

And then in Sunspot's model, we would like to force it to use the new field:

class Tag
 .
 .
 .
 
  include Sunspot::Mongoid
  searchable do
    text :name, :as => :name_ac

  end
 
end

Sunspot Solr startup problem on Windows

There is an error on Windows because command-line arguments are escaped in a wrong way.

This fix works for me:http://www.trdev.co.uk/2011/07/03/sunspot-and-solr-search-on-windows-7-errors/

Basically, we just go to sunspot gems in Ruby and change this file:

sunspot/lib/sunspot/server.rb

and do this patch

       command << "-Xms#{min_memory}" if min_memory

       command << "-Xmx#{max_memory}" if max_memory

       command << "-Djetty.port=#{port}" if port

-      command << "-Dsolr.data.dir=#{solr_data_dir}" if solr_data_dir

-      command << "-Dsolr.solr.home=#{solr_home}" if solr_home

-      command << "-Djava.util.logging.config.file=#{logging_config_path}" if logging_config_path

+      command << "-Dsolr.data.dir=\"#{solr_data_dir}\"" if solr_data_dir

+      command << "-Dsolr.solr.home=\"#{solr_home}\"" if solr_home

+      command << "-Djava.util.logging.config.file=\"#{logging_config_path}\"" if logging_config_path

       command << '-jar' << File.basename(solr_jar)

       FileUtils.cd(File.dirname(solr_jar)) do

-        exec(Escape.shell_command(command))

+        exec(command.join(" "))

       end

     end

Rename muliple files, remove prefix on linux

for f in $(ls uploads__*); do mv "${f}" "${f#uploads__}";done

Rails 3 strangely turns the 8th field of MySql into a BigDecimal

Please see my post: http://stackoverflow.com/questions/6646507/activerecord-in-rails-3-0-3-turns-the-8th-field-of-mysql-into-a-bigdecimal-how-t

I still cannot solve this problem.


-----------

I can solve it now. It is just a mismatch version between MySQL database and client.

Rails 3, RoutingError crashes the server

I have encountered this weird behavior. It took me 6 consecutive hours to solve it by debugging the whole Rails code.

In this post, someone suggests that we switch from WEBrick to Mongrel.

I have found out that the code piece which causes the problem is (It is located in C:\Ruby192\lib\ruby\gems\1.9.1\gems\railties-3.0.9\lib\rails\rack\log_tailer.rb):

module Rails
  module Rack
    class LogTailer
      .
      .
      .

      def tail!
        @file.seek @cursor

        if !@file.eof?
          contents = @file.read
          @cursor = @file.tell
          $stdout.print contents
        end
      end
    end
  end
end

If you blank this method, WEBrick works fine again. I have done an intensive test on it with a lot of RoutingError thrown.

You can use this patch. Put it in the environment file:

module Rails
  module Rack
    class LogTailer

      def tail!

      end
    end
  end
end

The downside of this is that you won't see debug messages on your console.

To bypass this problem, you can use log4r to output debug messages on console instead.

Work like a charm for me.

Install MySql2 properly on Windows for Rails 3

You will need to configure your Bundle to use some flags by:

 bundle config build.mysql2 --with-mysql-lib="C:\xampp\mysql\lib\opt" --with-mysql-include="C:\xampp\mysql\include"

If you cannot find lib\opt and include folder, please install MySql Connector/C. The folders will be there.

Then you must use mysql2 version 0.2.6 for Windows. Other versions do not work.

In your Gemfile, you should have this:

gem 'mysql2', '0.2.6'

And then you can run:

bundle install (or update)

This will solve all your problems, including "incompatible character encodings: UTF-8 and ASCII-8BIT".

Ruby global variable cheat sheet

$name program-defined global variable
$! latest error message
$@ location of error
$_ string last read by gets
$. line number last read by interpreter
$& string last matched by regexp
$~ the last regexp match, as an array of subexpressions
$n the nth subexpression in the last match (same as $~[n])
$= case-insensitivity flag
$/ input record separator
$\ output record separator
$0 the name of the ruby script file
$* the command line arguments
$$ interpreter's process ID
$? exit status of last executed child process

Rails plugin init.rb technique

Never ever do this in your init.rb

  require 'db_migration'

Sure, Rails will automatically look into your plugin lib folder. But if another file with identical name is loaded before (from other plugin), then Rails won't look anywhere. It'll just grab that file.

Of course this is a behaviour that you don't want. You'd like each plugin to have its own space. Then, do this:

  File.dirname(__FILE__)+'/lib/db_migration'

Rails Patch to obtain a current template

There are 3 types of rendering: Normal, Partial, and Layout.

We have to hook all 3 types.

The template is stored in @last_template.

module ActionView::Rendering 
  alias_method :_render_template_original, :_render_template
  def _render_template(template, layout = nil, options = {}) 
    @last_template = template
    _render_template_original(template, layout, options)
  end
end

module ActionView::Layouts
  alias_method :_render_layout_original, :_render_layout
  # Contains the logic that actually renders the layout.
    def _render_layout(layout, locals, &block) #:nodoc:
      @last_template = layout
      
      _render_layout_original(layout, locals, &block)
    end
end

module ActionView::Partials
  alias_method :_render_partial_original, :_render_partial

  def _render_partial(options, &block) #:nodoc:
    
    prefix = self.controller_path unless options[:partial].include?(?/)
    @last_template = self.find_template(options[:partial], prefix, true)
    
    _render_partial_original(options, &block)
    
  end
  
end

Stupid Rails3 with Missing template and render_to_string on partial view

render :json=>{:ok=>true ,:new_row=>render_to_string(:partial=>"row",:locals=>{:entity=>entity,:field_set=>SHOW_FIELDS,:is_new=>false}), :entity=> entity}

This gives you a "Missing template" error, even the view exists. I have found that this is because an AJAX request on Rails does not include the view format :html.

In order to fix it, just append .html.

render :json=>{:ok=>true ,:new_row=>render_to_string(:partial=>"row.html",:locals=>{:entity=>entity,:field_set=>SHOW_FIELDS,:is_new=>false}), :entity=> entity}

And it will work.

Stupid Rails 3.0.3

mysql2 version 0.2.6 is the only version compatible with Windows :S

Finally -- Redis

I have been playing around with Redis for a couple days.

Finally, I got a design that works very well and very fast.

This is the general design when using Redis.

For example, if we have post, which has fields: title, created_date, user_id.

We can store it like this

post:$id:title --> "Example Title"
post:$id:created_date --> "2011-10-10"
post:$id:user_id --> 5

But when you want to query it, please use mget .

You will get the result at once.

Redis has a big problem with network latency and we can use mget to minimize the number of queries.

MongoDB

It is just a RDBMS without static schema...

Database optimization -- Never ever forget to do it

Auto-login-after-action architecture

We should always create a pending table:

[data, status, unique_key]

The column 'data' is for all the necessary data for an action encoded in JSON.
The column 'status' is for checking if it is done or not.
The column 'unique_key' is to identify the returning user.

With this way, the pending table is independent from the action table.

Top of the week architecture

I have come up with an architecture which is quite ok.

First, we shall have a table:

[guid, date, hour, entity_id, score]

The column 'guid' is the primary key of the table, which is 'date' concatenated with 'hour'.

When the score is incremented or decremented, we also update this table. Here is the crucial technique for updating:

INSERT INTO `hour_counts` (`guid`,`date`,`hour`,`entity_id`,`score`) VALUES ('#{guid}','#{time}','#{hour}','#{entity_id}','#{score}') ON DUPLICATE KEY UPDATE `score`=`score` + 1;

With this SQL statement, the record is automatically created if it is not there. If it is there, then we just increment the score.

Since MySQL handles concurrency for us, we can be sure that there is no duplicate record.

That is pretty cool.

Do not build a website that solves a specific task.

Your market will be too small, and nobody will come to use your website.

This is a lesson learned from CollegeSwap. The market is too small and it is so specific that nobody comes to use it.

Well, of course, build it if your marketing plan is a very clever one.

Lesson from FriendMage

Never ever create anything, any website, or any software:

you do not feel passion about.

OR

you haven't been used similar websites for a long time

OR

It involves too many skill sets, which are out of your leagues.

The reason is simple. It it because you will not maintain it and you will not improve it. More importantly, you will not know what to be improved, really.

People do not like to login

This is a proof that people do not like to login. Or they do not like to explicitly press the login button.

You can see that the yellow line is peaked at 5:00. That point is the number of guests who tried to post comments, but it failed though because they have to login before posting comments.

At 6:00 and 7:00, the comment box is hidden for ones who haven't logged in.

We can see that the red line is still the same. (The red line is the number of comments posted by members)

The conclusion is that when we force people to login before posting comments, they just leave!

We have to use Full-text indexing at some point

I have found that the blog describes very well about Full-text indexing modules:

http://locomotivation.squeejee.com/post/109279130/simple-ruby-on-rails-full-text-search-using-xapian

http://locomotivation.squeejee.com/2008/07/15/mulling-over-our-ruby-on-rails-full-text-search-options

Database design for tag-based search

Ref: http://incarnate.ru/post/1439084341/database-design-for-tag-based-search#disqus_thread

Delayed Job cheatsheet

To add email sending to background job, use:

# NotificationMailer.deliver_welcome_user(@user)
# We do not use the above line anymore
NotificationMailer.send_later(:deliver_welcome_user, @user)

Or you can use a more customizable version:

# put this in lib/delayed_notification.rb
class DelayedNotification
  attr_accessor :user_id
  def initialize(user_id)
    self.user_id = user_id
  end

  def perform
   NotificationMailer.deliver_welcome_user(User.find(@user_id))
  end
end

# Add this where you were sending the mail earlier
Delayed::Job.enqueue DelayedNotification.new(@user.id)

Ref: http://8raystech.com/2009/2/28/background-job-processing-in-rails-with-delayed_job

Stop Aptana RadRails from generating .tmp (for HTML previewing)

It's quite annoying for us that Aptana RadRails keeps generating .tmp__... for HTML and ERB files that are previewed.

Therefore, in order to disable it:

1. Go to Preferences
2. Aptana->Editors->HTML->Preview
3. Uncheck 'Generate temporary files for browser previews'

Rails version 3 and 2 difference on processing web request

How to get Rails version

You can refer to

Rails::VERSION::STRING

It will return a version as string, e.g., 3.0.3

A service to start at boot time

We have to build a shell script with this line

# chkconfig: - 85 15

(Please google to see what it means)

Then we type

chkconfig <your_service_file_name> on

That's it.

Run background process with Capistrano

To run a background process with Capistrano is so difficult. I have tried few different ways and I found the working one.

It turns out that you need to use a command, nohup together with &, which is put in a shell script file.

Then in Capistrano file (deploy.rb), you'll need to use run the shell script with >/dev/null 2>&1.

And that's it.

-------

I have tried use :pty=>true option on the Capistrano's run command. It produces a very strange behavior. Sometimes the background process is started and sometimes it is not...

And without >/dev/null 2>&1, there will be an output, which blocks Capistrano from proceeding forward...

Fix: msvcr90.dll is either not designed to run ... error

Out of nowhere, my beloved Vaio has the error on msvcr90.dll. It just corrupted. Just like that.

The fix is really simple. Just replace the DLLs file.

mscvr90.dll is the files of Visual Studio 2008 redistributable.

Go to C:\Program Files\Microsoft Visual Studio 9.0\VC\redist\x86\Microsoft.VC90.CRT...

Copy all 3 DLLs to the destination folder. and that's it.. it works like a charm.

Full error reporting on Rails production

Just comment all these lines:

# The production environment is meant for finished, "live" apps.
# Code is not reloaded between requests
config.cache_classes = true

# Full error reports are disabled and caching is turned on
config.action_controller.consider_all_requests_local = false
config.action_controller.perform_caching             = true
config.action_view.cache_template_loading            = true

And add these lines, instead:

config.cache_classes = false

# Log error messages when you accidentally call methods on nil.
config.whiny_nils = true

# Show full error reports and disable caching
config.action_controller.consider_all_requests_local = true
config.action_view.debug_rjs                         = true
config.action_controller.perform_caching    

Bad URI with oauth token

Bad URI error occurs when we do OAuth verification. (I found it because of Facebook connect)

This is because the OAuth token contains a special character, precisely, |. I think there can be other characters that cause Bad URI error.

Moreover, I have found that the Bad URI only occurs with WEBRick, a webserver provided by Ruby.

Therefore, in order to fix it, I had to modify Ruby source code.

Here is what I have done:

In the file httpresponse.rb, which resides in C:/Ruby187/lib/ruby/1.8/webrick (This depends on where you installed Ruby)

Around the line 164:

      # Location is a single absoluteURI.
      if location = @header['location']
        if @request_uri
          @header['location'] = @request_uri.merge(location)
        end
      end

Add URI.encode() around location before @request_uri.merge()

# Location is a single absoluteURI.
      if location = @header['location']
        if @request_uri
          @header['location'] = @request_uri.merge(URI.encode(location))
        end
      end

And that's it.

I'm gonna fix this in the Ruby open source project also. This will be my first open source participation, yeahhhh.

Communications link failure (MySQL)

When you find this exception:

com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The cause is that you cannot to MySQL server. Simply, try:

telnet 127.0.0.1 3306

in order to verify that it does not work.

My problem is that I bind MySQL server to a specific address.

I fix it by binding MySQL server to 0.0.0.0 (all network interface).

Get POST parameters from Java HTTP Server

It turns out that it is not very simple to do so.

First, Create a class name ParameterFilter

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */

package friendposter;

import com.sun.net.httpserver.Filter;
import com.sun.net.httpserver.Filter.Chain;
import com.sun.net.httpserver.HttpExchange;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.URI;
import java.net.URLDecoder;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

/**
 *
 * @author Tanin
 */
public class ParameterFilter extends Filter {

    @Override
    public String description() {
        return "Parses the requested URI for parameters";
    }

    @Override
    public void doFilter(HttpExchange exchange, Chain chain)
        throws IOException {
        parseGetParameters(exchange);
        parsePostParameters(exchange);
        chain.doFilter(exchange);
    }

    private void parseGetParameters(HttpExchange exchange)
        throws UnsupportedEncodingException {

        Map parameters = new HashMap();
        URI requestedUri = exchange.getRequestURI();
        String query = requestedUri.getRawQuery();
        parseQuery(query, parameters);
        exchange.setAttribute("parameters", parameters);
    }

    private void parsePostParameters(HttpExchange exchange)
        throws IOException {

        if ("post".equalsIgnoreCase(exchange.getRequestMethod())) {
            @SuppressWarnings("unchecked")
            Map parameters =
                (Map)exchange.getAttribute("parameters");
            InputStreamReader isr =
                new InputStreamReader(exchange.getRequestBody(),"utf-8");
            BufferedReader br = new BufferedReader(isr);
            String query = br.readLine();
            parseQuery(query, parameters);
        }
    }

     @SuppressWarnings("unchecked")
     private void parseQuery(String query, Map parameters)
         throws UnsupportedEncodingException {

         if (query != null) {
             String pairs[] = query.split("[&]");

             for (String pair : pairs) {
                 String param[] = pair.split("[=]");

                 String key = null;
                 String value = null;
                 if (param.length > 0) {
                     key = URLDecoder.decode(param[0],
                         System.getProperty("file.encoding"));
                 }

                 if (param.length > 1) {
                     value = URLDecoder.decode(param[1],
                         System.getProperty("file.encoding"));
                 }

                 if (parameters.containsKey(key)) {
                     Object obj = parameters.get(key);
                     if(obj instanceof List) {
                         List values = (List)obj;
                         values.add(value);
                     } else if(obj instanceof String) {
                         List values = new ArrayList();
                         values.add((String)obj);
                         values.add(value);
                         parameters.put(key, values);
                     }
                 } else {
                     parameters.put(key, value);
                 }
             }
         }
    }
}

Then, when start a server, please add ParameterFilter as a filter:

HttpServer server = HttpServer.create(new InetSocketAddress(PORT),100);

                HttpContext context = server.createContext("/", new FriendPosterServer());
                context.getFilters().add(new ParameterFilter());

                server.setExecutor(null); // creates a default executor

                server.start();

Then, in the handler, you may obtain parameters like this:

public void handle(HttpExchange t) throws IOException
{
...
Map params = (Map)t.getAttribute("parameters");
...
}

Java to do HTTP Post and read response's content

public void postToCallbackUrl()
    {

        
        int retry_count = 0;

        while (retry_count < 3)
        {
            try
            {
                // do HTTP request
                List<NameValuePair> formparams = new ArrayList<NameValuePair>();
                formparams.add(new BasicNameValuePair("print_image_url", printImageUrl));
                formparams.add(new BasicNameValuePair("signature_image_url", signatureImageUrl));
                formparams.add(new BasicNameValuePair("order_key", orderKey));

                UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formparams, "UTF-8");

                HttpPost httppost = new HttpPost(callbackUrl);
                httppost.setEntity(entity);

                HttpClient httpclient = new DefaultHttpClient();
                HttpResponse response = httpclient.execute(httppost);

                logger.debug("Got response:" + IOUtils.toString(response.getEntity().getContent(), "UTF-8"));

                break;
            }
            catch (Exception e) {
                retry_count++;

                logger.error("Post error",e);
            }
        }
    }

How to migrate database in production environment

rake db:migrate RAILS_ENV="production"

Ruby cannot find installed gems

Problem

When we run any ruby file with ruby command, it won't be able to find all installed gems.

Solution

Add require 'rubygems' at the top of the file

Setting up multiple IP addresses on a single NIC (Linux)

Copied from: http://linuxhelp.blogspot.com/2005/05/setting-up-multiple-ip-addresses-on.html
(I just copy it in case it got lost)

In linux, you can bind multiple IP addresses on a single NIC. This is usually done in case you are using your linux machine as a web server and is hosting multiple domains and you want to bind each domain to a unique IP address. This is how it is done.

Let us assume that you already have a NIC which is bound with a static IP address. Then you will have a file called /etc/sysconfig/network-scripts/ifcfg-eth0 .My ifcfg-eth0 file has the following entries:

# File: ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.0.1
NETMASK=255.255.255.0
BROADCAST=192.168.0.255
NETWORK=192.168.0.0
HWADDR=00:80:48:34:C2:84

Now to bind another IP address to the same NIC, I create a copy of the above file ifcfg-eth0 and name it as ifcfg-eth0:1

cd /etc/sysconfig/networking-scripts
cp ifcfg-eth0  ifcfg-eth0:1

Now just change the values of the DEVICE and IPADDR in the file as follows:

# File: ifcfg-eth0:1
DEVICE=eth0:1
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.0.5
NETMASK=255.255.255.0
BROADCAST=192.168.0.255
NETWORK=192.168.0.0
HWADDR=00:80:48:34:C2:84

And lastly, restart the networking service. If you are using RedHat, then it is as simple as :

service network restart

Facebook app to don't show scrollbar at all

In facebook app setting, just set the IFrame scrollbar option.

That's it.

Ruby has a weird mechanism of escaping a single quote and a double quote

Just don't try to escape a single quote. It's super complicated.

With a double quote, it is a little bit easier. Here's the cheat sheet:

'test " test'.gsub('"', '\\"')
# we got 'test \\\" test'

'test " test'.gsub('"', '"')
# we got 'test \" test'

Amazing, huh ?

How to make a slided page

In order to make a slided page. We need to have a frame with these minimal settings;

...

Inside of the frame, you need a span that contains the whole panel:

    
      ....
    

And each page is a span with these settings:

display:inline-block; // it will align on the same line
vertical-align:top; // it will align on the top
white-space:normal; // override white-space property because, in firefox, a child inherits white-space from its parent.

That makes:

In order to slide a page, we modify margin-left of the inner_frame

Mouseover-capturing trick when a span has children

Problem


The problem occurs when a span has children but you want to capture mouseover event over the span

Solution


The most elegant solution is to
1. add position:relative to the span
2. add a child span

3. Attach all events to the newly added child span instead.

It will be on top; if it doesn't, add z-index.

Navigate away with a link before finish Ajax request (jQuery)

And it'll produce an error.

Anyway, there is a way to detect whether the failure comes from navigating away or other causes.

$.ajax({
    type: "POST",
    url: '/test',
    cache: false,
    data: {
     
    },
    success: function(data){
     
    },
    error: function(req, status, e){
     if (req.status == 0) return;
     alert('Cannot connect to the server. Please try again later.');
    }
   });

On the line 12 is the trick

Encode URL and HTML using Javascript

We use native function of Javascript:

encodeURIComponent("http://www.yahoo.com");
// We'll get "http%3A%2F%2Fwww.yahoo.com"

Encode URL and HTML with Ruby

We simply use:

require 'cgi'
CGI.escape("http://www.yahoo.com")

Rails Critical Section through MySQL

The Problem

We have problem with agreeing with a comment in Squeks. If the same user agrees on the same comment at the same time, then 2 identical records of comment_agree will be created.

Therefore, we need some kind of critical section in order to prevent that situation.

The Solution

We use Named Lock provided by MySQL. You can use these two statements:

SELECT GET_LOCK('lock1',10);
SELECT RELEASE_LOCK('lock1');

Anyway, it's more convenient to build a class in order to deal with this:

class Lock
  def self.lock(name)
    return ActiveRecord::Base.connection.execute("SELECT GET_LOCK('#name}',60);").fetch_row[0]
  end

  def self.release(name)
    ActiveRecord::Base.connection.execute("SELECT RELEASE_LOCK('#{name}');")
  end

  def initialize(*name)
    @lock_name = Lock.generate_name(name)
  end

  def synchronize(&block)

    Lock.lock(@lock_name) rescue (raise "Getting Lock #{@lock_name} timeout")

    block.call()

    while true
      Lock.release(@lock_name) rescue next
      break
    end

  end

  private
  def self.generate_name(names)
    names.map{ |t| t.to_s}.join('--')
  end
end

Welcome to whoWish knowledge-based system

This blog is for keeping programming techniques or installation techniques.

It is embedded with code snippet feature. For example:

// Comment
public class Testing {
public Testing() {
}
 
public void Method() {
/* Another Comment
on multiple lines */
int x = 9;
}
}

$(real_obj).blur(function()
       {
           if ($(real_obj).val() == "")
           {
               $(default_obj).show();
      $(real_obj).hide();
           }
       });

 user = $facebook 
 user = get_facebook_info(params[:user_id]) if params[:user_id]

So what else I need to have?

This is what I have:
- Good IDE
- Automated deployment
- Source Control
- Project Management for communication
- Logging practice

We needs:
- Knowledge-based system

And what else?

Current log viewer

We use LogExpert, which is free and has a nice highlight feature.

But I haven't tested it extensively in order to test if it can handle high-load of information or not.

It also has a tail feature.

So this tool is ok.

Best Web-based hosted project management tools

I have tried several project management tools. I have found out that Wrike is the best one (at least, for me). These are the PM tools that I have tried:

1. Basecamp - It's nice, but a task cannot be attached with files, and it does not have bug reports area.
2. 5pm - The UI is so uncomfortable and it does not have bug reports area.
3. AceProject - This one is not-well-designed. It's very difficult to use.
4. ProjectPlace - Too expensive. It's 1 person for 27 dollars per month per project.
5. CentralDesktop - a task cannot be attached with files

Wrike has everything:
- A task can be attached with files
- You can create any folder and any sub-folder. This means that I can create a sub-folder 'bugs' under any project.
- You can have as many folders as you want because Wrike charges according to number of users
- UI is generally good. It is easy to add task and assign to people.

And I feel like I start acting like a real CTO ...

Logging Best Pratice (Draft #0)

There are 5 levels of logging to utilize: DEBUG, INFO, WARNING, ERROR, and FATAL.

DEBUG
We should log these 4 entities:

1. Entry point of a method with parameters
2. Exit point of a method with returned values
3. the condition of a if-else and which branch it results in
4. every iteration of a loop with critical data

INFO
The general purpose of INFO level is to provide contextual information.

For example, you should log when a user logs in and also log the result of logging in. (Please also provide parameters information and results if it does not incur too much CPU cost).


WARNING
We should log some event as a warning when there is something wrong but it is not the code's fault.

For example,
1. user calls for a non-existing entity (the id is invalid).
2. Invalid status or unexpected status.
3. User is not logged and he/she is not supposed to have access to this method.

The general purpose of WARNING level is to reveal security problems.

ERROR
We should log all exceptions thrown in try-catch blocks. Please do include traces in the log.

FATAL
At FATAL level, we only log when the server is going to crash. I cannot think of any event that we have to log right now.

Besides, the server automatically logs errors for us when a fatal error occurs.

Log4r + Rails

There is only one way to integrate log4r with Rails.

If you do the following, you will encounter a very strange error, which is raised from "render :partial" because it cannot find outputters.

config.logger = Log4r::Logger.new("Application Log")
console_format = Log4r::PatternFormatter.new(:pattern => "%l:\t %m")
config.logger.add Log4r::StdoutOutputter.new('console', :formatter=>console_format)

file_format = Log4r::PatternFormatter.new(:pattern => "[ %d ] %l\t %m")
config.logger.add Log4r::FileOutputter.new('fileOutputter', :filename => RAILS_ROOT+"/log/"+RAILS_ENV+".log", :trunc => false, :formatter=>file_format)

config.log_level = :debug

SO DON'T DO THAT

Instead, please do this:

1. In environment.rb, put in this line:

... require File.join(File.dirname(__FILE__), 'boot')

require File.expand_path(File.dirname(__FILE__) + "/logger")

Rails::Initializer.run do |config| ...

2. Create logger.rb in config folder with these lines:

--------------------------

# Configure logging.
# We use various outputters, so require them, otherwise config chokes
require 'log4r'
require 'log4r/yamlconfigurator'
require 'log4r/outputter/fileoutputter'
require 'log4r/outputter/datefileoutputter'
require 'log4r/outputter/emailoutputter'


cfg = Log4r::YamlConfigurator
cfg['RAILS_ROOT'] = RAILS_ROOT
cfg['RAILS_ENV'] = RAILS_ENV

# load the YAML file with this
cfg.load_yaml_file("#{RAILS_ROOT}/config/log4r.yaml")
RAILS_DEFAULT_LOGGER = Log4r::Logger['default']
RAILS_DEFAULT_LOGGER.level = (RAILS_ENV == 'development' ? Log4r::DEBUG : Log4r::INFO)

if RAILS_ENV == 'test'
Log4r::Outputter['stderr'].level = Log4r::OFF
RAILS_DEFAULT_LOGGER.add( Log4r::Outputter['stderr_test'] )
end

if RAILS_ENV == 'production'
Log4r::Outputter['standardlog'].level = Log4r::OFF
Log4r::Outputter['stderr'].level = Log4r::OFF
else
Log4r::Outputter['email'].level = Log4r::OFF
end


--------------------------

3. Create log4r.yaml in config folder with these lines:


--------------------------

# *** YAML2LOG4R ***
log4r_config:
# define all loggers ...
loggers:
- name : default
level : DEBUG
additive : 'false'
trace : 'true'
outputters:
- stderr
- datefilelog
- standardlog
- email

# define all outputters (incl. formatters)
outputters:
- type : StderrOutputter
name : stderr
level : WARN
#only_at :
# - INFO
# - WARN
# - FATAL
formatter:
date_pattern: '%y%m%d %H:%M:%S'
#pattern : '%d %l: %m '
pattern : "[%l] %d :: %m"
type : PatternFormatter

- type : StderrOutputter
name : stderr_test
level : ERROR
formatter:
date_pattern: '%y%m%d %H:%M:%S'
# Added 'M' and 't' here - they are slow, but we're not calling unless we need to
pattern : "[%l] %d :: %M :: %t"
type : PatternFormatter

- type : DateFileOutputter
name : datefilelog
level : DEBUG
date_pattern: '%Y%m%d'
trunc : 'false'
dirname : "#{RAILS_ROOT}/log"
formatter :
date_pattern: '%y%m%d %H:%M:%S'
#pattern : '%d %l: %m '
pattern : "[%l] %d :: %m"
type : PatternFormatter

- type : FileOutputter
name : standardlog
level : DEBUG
trunc : 'false'
filename : "#{RAILS_ROOT}/log/#{RAILS_ENV}.log"
formatter :
date_pattern: '%y%m%d %H:%M:%S'
pattern : "[%l] %d :: %m"
type : PatternFormatter

- type : EmailOutputter
name : email
level : ERROR
server : smtp.example.com
port : '25'
#domain : smtp.example.com
authtype : none
#authtype : basic
#acct : dansketcher
#passwd : passwd
subject : 'Application Error'
from : error-handler-person@example.com
to : error-handler-person@example.com
immediate_at: "FATAL, ERROR"
formatfirst : 'true'
formatter :
date_pattern: '%y%m%d %H:%M:%S'
# Added 'M' and 't' here - they are slow, but we're not calling unless we need to
pattern : "[%l] %d :: %M :: %t"
type : PatternFormatter


--------------------------

Done !

Facebook Connect with redirect_uri problem

It's obvious that redirect_uri cannot contain the '?' (question mark) character.

IE + ajax + HTTP 408

We (Nilobol and I) have found that ajax requests on IE occasionally fail.

Here is what we have found:
- The request is sent out
- But we never get a reply back from the server
- The server actually sees the request but it response as HTTP 408 (request timeout)

Right now we are on this problem. We have found some thread saying that because on the header Connection: keep-alive. IE has problem with its implementation.

We are changing all ajax request to be Connection: close (both on clients and server).

We are confirming the result, but it looks good now.

Cookies enabled on IE (P3P)

I have been encountered the problem of using cookies on IE for a while.

The problem surfaces because a facebook app is run in an iFrame. Therefore, IE considers it as a third-party cookie.

In IE, one can choose a privacy option to be high. This means that, without a privacy policy, IE won't allow you to use cookies.

* Privacy Policy is W3C standard, which only IE takes it seriously.

We have found the way to make cookies work even users set the privacy to be the second-highest option. (Even hotmail does not work.)

We have to use these compact polciies: NOI COR PSA OUR IND OTC

Here are their meanings:

NOI = Web Site does not collected identified data.
COR = Errors or wrongful actions arising in connection with the privacy policy will be remedied by the service.
PSA = Information may be used to create or build a record of a particular individual or computer that is tied to a pseudonymous identifier, without tying identified data
OUR = Ourselves and/or entities acting as our agents or entities for whom we are acting as an agent. (I don't know what this means)
IND = Information is retained for an indeterminate period of time. The absence of a retention policy would be reflected under this option. Where the recipient is a public fora, this is the appropriate retention policy. (This is to use never-expired cookies)
OTC = Other types of data not captured by the above definitions. (This says that we won't contact user in real world or online world (which is a lie because we contact users by their emails)

Basically, these compact policies say that we don't identify users, which is a lie because we have users' facebook id.


But, hey, it works ...

Automated deployment setup (Capistrano)

It is really tricky in order to automate the deployment. First, install Capistrano on your local machine:

gem install capitrano

then,

capify /project_dir

It will create a file deploy.rb under config folder. Now make it look like this:

set :application, "CollegeSwap"
set :domain, "72.26.225.10"
set :repository, "git@github.com:tanin47/CollegeSwap.git"

set :use_sudo, false
set :deploy_to, "/#{application}"
set :scm, "git"

set :user, "deploy"

set :branch, "master"

role :app, domain
role :web, domain
role :db, domain, :primary => true

# If you are using Passenger mod_rails uncomment this:
# if you're still using the script/reapear helper you will need
# these http://github.com/rails/irs_process_scripts

namespace :deploy do
task :start do ; end
task :stop do ; end
task :restart, :roles => :app, :except => { :no_release => true } do
run "#{try_sudo} touch #{File.join(current_path,'tmp','restart.txt')}"
end
end

Now go to your server and clone the project. On the local computer, you can type 'cap deploy' to re-deploy the project.

There will be some issue about permission. You can try setting the deploy user and nignx user to be the same user. That's probably the simplest way.

Or you can set them to be in the same group. Give the access to the group instead by 'chown -R :nginxgroup /CollegeSwap'.

Then, you need to type 'chmod g+s /CollegeSwap'; every file created under this dir will inherit permission from its parent.

And that's it. you're good to go.

One last thing, do not forget to 'chmod -R 777 /CollegeSwap'

Nginx: 403 Forbidden

When you found this weird error while using Nginx with Passenger, there can be of 2 causes.

First, try playing with permission stuff.

If that does not help, it might be because of your config file.

you config file should not look like this:

server {
listen 80;
server_name wikicards.org;
access_log /var/log/nginx/wikicards.access.log;
passenger_enabled on;

location / {
root /var/apps/wikicards/public;
index index.html index.htm;
}
}

* location and passenger_enabled on should not be in the same block. Remove the location block and that's it.

Nginx installed !

And now we have the fastest web server running on our machine.

It's pretty simple. You don't need to install Nginx first.

just install Passenger with 'gme install passenger', then type 'passenger-install-nginx-module'

It will tell you to do something with the config file. Just don't forget to do it.

And that's it.

I've just read that Nginx is much faster than Apache

I'm switching to it right now.

--------------------------------------

I cannot get Nginx to run faster than Apache. Maybe I don't know the appropriate configuration for it.

Anyway, right now I'm using Heroku instead.

Apache 2.2 + Phusion Passenger

Apache 2.2 + Phusion Passenger is awesome. I have been using WEBrick for a while. It was friggin' slow. There are also other awesome alternatives: Thin, Mongrel, and nginx with unicorn.

Nginx + Unicorn is better at serving customer with high-speed internet connection. Actually, Unicorn is said to strictly serve local clients only, e.g., another program within the same computer. Therefore, Unicorn is good for a Rack-based application. You can read its well-written philosophy here. It also states why Nginx, not Apache, is the only webserver that is appropriate for Unicorn.

(There is an HTTP server, Rainbow, which strictly serves remote, low-speed client.)

So, Unicorn is not our appropriate solution.

Thin is said to be very faster. Nevertheless, this benchmark said that Passenger is the fastest web server among Thin and Mongrel.

(And some people have said that installing Mongrel is pains in the ass)

I haven't verified it myself, and I won't do it. I'm just gonna take it for grant :D

---------------------------------------------------

Ok, so this is how you install Apache with Passenger.

First, install Apache. This is pretty damn easy. If you cannot do it yourself, I suggest you should change your job.

Second, install passenger. (You need gem, just don't do it manually) type 'gem install passenger'.

Then, you type 'passenger-install-apache2-module'. If it does not detect your apache, then try to set the option '--apxs2-path'.

On the screen it will tell you to copy some text to Apache configuration file and how to point to the rails application. That's it !

libmysql for Ruby 1.8.6 must be a special one

You cannot just get it from MySQL's bin folder. It simply does not work. You need a specific libmysql.dll.
(I have it in my gmail. You can request from me.)

2 Hackathons reflected

So many mistakes we have done. Most of the mistakes came from me. It's worth to list all the mistakes that we have done.

The first big mistake that we have done is to greedily draw the scope. Our scope of HopIn was initially friggin' big, vertically and horizontally. Right now I have a theory that a scope has two dimensions on it: Vertical and Horizontal. Scope should be devised toward vertical direction, at least, in the beginning.

To extend scope vertically means that we are building the application toward our goal regardless of how many people we polarize. To extend scope horizontally means that we are building the application to support more people. An example is: when we build an application for trading; if we go for supporting more numbers of currencies, that's extending the scope horizontally. If we go for supporting 'shipping cost calculation', that's veritical extension.

Within a day, we can only finish a really small vertical scope. The lesson here is to draw a small vertical scope. If the scope is to wide and tall, phrase it.