Sentiment analysis and adaptive phrase match learning with PHP

I have recently released a sentiment analysis PHP class under the GPL licence that both analyses the sentiment of text as well as matches text with previously analysed phrases that are positive, negative or neutral. In other words, it learns from your input and becomes more accurate over time. Below I will outline the general concept.

Simple example

include ('sentiment_analyser.class.php');
$sa = new SentimentAnalysis();
$sa->initialize();
$sa->analyse("Thank you. This was the best customer service I have ever received.");
$score = $sa->return_sentiment_rating();
var_dump($score);

Concept

This class serves three purposes:

  1. Estimate the sentiment for a string based on emotion words, booster words, emoticons and polarity changers
  2. Allow you to save analysed data into positive, negative or neutral datasets
  3. Identify if we have any phrase matches on previously analysed positive, negative and neutral phrases

Should there be any high quality phrase matches, it would take precedent over the sentiment analysis and return the phrase match rating instead.

Sentiment Analysis

Strings are broken into tokenised arrays of single words. These words are analysed against TXT files that contain emotion words with ratings, emoticons with ratings, booster words with ratings and possible polarity changers.

A score is then calculated based on this analyse and this forms the “Sentiment analysis score”.

Phrase Analysis

This function is key to identifying whether the phrase in questions can be compared to phrases that we have analysed and stored before. It uses Levenshtein distance to calculate distance between 4,5,6,7,8,9 and 10 word length phrases against the dataset we already have. We also make use of PHP’s similar_text to double verify proximity.

This means that the more phrases we have analysed previously improves the entire dataset and allows phrases to be more accurately scored against historical data.

  1. The phrase is broken up into ngram lengths
  2. The array is reverse sorted so we compare 10 word length phrases first, then 9, and so on
  3. Phrases are matched against positive, negative and neutral phrases in the relevant TXT files
  4. Only matches that meet the minimum levenshtein_min_distance and similiarity_min_distance are kept

The Evolution of myScoop

It has been over 4 years since I created myScoop, the once called “the real time South African blog aggregator”. Let’s face it, aggregation is dead. As with all things in life, change should be the only constant and this is the reason myScoop has been completely overhauled into something new – a premium blog network that offers bloggers the opportunity to monetise their blogs and offers advertisers the opportunity to get their message out to millions of South Africans in a relevant and unique manner.

The idea

I have been playing with this idea for quite some time now. Just like AdDynamo’s “sponsored tweets” concept, the concept of “paid blogging” is relatively new to South Africa and there have been a few discussions around it over the past year. I have been working full tilt to get the system up and running and I’m finally proud to announce the relaunch of myScoop.

How it works

It’s quite a simple tool at the moment. Bloggers have their choice of which campaigns they would like to be a part of and get paid according to how popular their blog is. Advertisers create campaigns within myScoop and then choose which blogs they would like to have writing about their brand, product or service. For ethical reasons, all sponsored posts need to be fully disclosed on the blogs. All of this is controlled through the myScoop App.

myScoop has over 1000 South African blogs registered giving potential advertisers the luxury of choice as to when and how their message is spread throughout the millions of South African internet users.

 

Introducing PingPong, site uptime and performance monitor

I’ve been feverishly working on my new startup, PingPong. It’s a website uptime and performance monitoring tool that sends you an SMS and Email as soon as your website goes down.

I believe this tool to be absolutely vital for any online business, agencies, and e-commerce websites. If you’re making money via a website or have a strong online presence it’s essential to reduce the amount of time your website is offline. It’s inevitable that your site will go down but knowing when it happens allows you to act quicker and get your site operational again. Hence the reason for creating PingPong.

PingPong is a site monitoring tool that:

  • Alerts you via SMS & email when your website(s) go down
  • Provides performance metrics
  • Provides error logs to help you debug your website downtime

There’s a few benefits of having such a tool:

  • If you’re in ecommerce and you’re site goes down, you obviously lose money.
  • If you are running AdWords campaigns and your site goes down, all your ads run the risk of being disapproved.
  • If you’re an agency, it’s helpful to know that a client’s site is down before they know. This allows you to act quicker.

PingPong is still very new and there is a host of features I plan on releasing over the next few months:

  • Automatic weekly/monthly reports (emailed to you)
  • Better, more modern user interface
  • More “checks” (such as checking if your mail server is up, checking if your MySQL server is up, etc)
  • Public page (allow others to see your downtime and performance reports)

I’d love to hear your thoughts on my new startup.

Evernote hacked: 50 million user accounts compromised

EvernoteEvernote, the service and app that allows you to record anything about your life, business or ideas recently sent out an email stating that they will be carrying out a service-wide password reset due to suspicious activity on their servers. Although they mention that they are taking precautionary measures to ensure your account is safe, they do also mention that your user information has been compromised. Oh great.

Dear Evernote user,

Evernote’s Operations & Security team has discovered and blocked suspicious activity on the Evernote network that appears to have been a coordinated attempt to access secure areas of the Evernote Service.

As a precaution to protect your data, we have decided to implement a password reset. Please read below for details and instructions.

In our security investigation, we have found no evidence that any of the content you store in Evernote was accessed, changed or lost. We also have no evidence that any payment information for Evernote Premium or Evernote Business customers was accessed.

The investigation has shown, however, that the individual(s) responsible were able to gain access to Evernote user information, which includes usernames, email addresses associated with Evernote accounts, and encrypted passwords. Even though this information was accessed, the passwords stored by Evernote are protected by one-way encryption. (In technical terms, they are hashed and salted.)

While our password encryption measures are robust, we are taking steps to ensure your personal data remains secure. This means that in an abundance of caution, we are requiring all users to reset their Evernote account passwords. Please create a new password by signing into your account on evernote.com.

After signing in, you will be prompted to enter your new password. Once you have reset your password on evernote.com, you will need to enter this new password in other Evernote apps that you use. We are also releasing updates to several of our apps to make the password change process easier, so please check for updates over the next several hours.

As recent events with other large services have demonstrated, this type of activity is becoming more common. We take our responsibility to keep your data safe very seriously, and we’re constantly enhancing the security of our service infrastructure to protect Evernote and your content.

There are also several important steps that you can take to ensure that your data on any site, including Evernote, is secure:

Avoid using simple passwords based on dictionary words
Never use the same password on multiple sites or services
Never click on ‘reset password’ requests in emails – instead go directly to the service
Thank you for taking the time to read this. We apologize for the annoyance of having to change your password, but, ultimately, we believe this simple step will result in a more secure Evernote experience. If you have any questions, please do not hesitate to contact Evernote Support.

The Evernote Team

I for one have an account with Evernote, let’s hope I don’t get an additional 100 spam emails sent to me every day over and above the 300 I already get.

How to evaluate and start any online business

Evaluate and start any online businessStarting any business and risking your normal monthly income can be a daunting task. Ask me, I have 3 kids and considerable monthly expenses. So what is required in order to start an online business with minimal risk? I’ll go through the basics below.

 

Evaluate your business idea

You may think your idea is amazingly brilliant but there’s a few tasks you need to perform in order to evaluate your potential business.

  • How many competitors are out there doing what you want to? Is the competition extensive?
  • Identify your competition and have a good look at how they are attempting to make money.
  • Ask yourself: what is your unique selling proposition? Do you really have something to offer?
  • Are you serious enough to follow this through?

Identify your market size

This is a critical step. You really don’t want to get knee deep in the business only to find out that no one is interested in what you have to sell.

Researching your market size is actually quite simple. There’s a few tools to help you with this but I would suggest using the Google AdWords Keyword Tool in order to find our just how many people are searching for what you want to sell. You can even filter down by the geographical area you are planning to serve. Once you have an estimated market size and the numbers still look promising enough for you to consider continuing, move forward to the next phase.

Validate your business model

This is the fun part. It doesn’t take a lot of capital to start an online business, unless your needing to keep stock but even then you can test your model before stocking up and spending all your capital on what could be a complete flop. Let me explain.

Suppose you want to start selling office chairs but you’re afraid that people may not buy from you. By following the below steps we can test your market quite easily.

Step 1: Set up the basics of your business
If you’re planning to sell products online, try uShop.co.za.

If you’re planning on selling your services online, try follow this tutorial on how create your own business website using WordPress. Remember to stick to South African hosting providers such as AmpleHosting and WebAfrica.

Step 2: Install visitor tracking software

You are going to need to know what your potential customers are doing on your website. Install free visitor tracking software such as Google Analytics in order to track visitor behavior  This will be essential in the tweaking phase.

Step 3: Test the market

In an attempt to test the market you could either use Google AdWords or Facebook Ads or both. Spend no more than R1000 on either. Both have fantastic user interfaces allowing first-time users to be able to set up a campaign within minutes. Follow these guides while you’re at it.

Step 4: Tweaking

After a few days of running your ad campaigns, you should have a decent idea of the amount of people wanting your product or service. Pay close attention to what the data is telling you in both your ad software (Google AdWords and/or Facebook) and Google Analytics. This step is crucial as a small change can be the difference between you achieving success or failure.

 

Go forth and conquer or cut your losses

After two weeks to one month of advertising online you should have a very good idea of your business’s potential. Ultimately, there’s three ways your business validation project could have turned out:

Dismal: You received no leads or phone calls while advertising. Unless you completely got your web design or advertising wrong, cut your losses and move on to your next idea.

Moderate: You received some good enquiries. What now? Keep building on your product, advertising and website in order to increase the amount of leads you can acquire online.

Holy smokes, Why didn’t I do this earlier: Quit your day job and take over the world.