Category Archives: Technology

MySwar Updates

We’ve been busy making MySwar better these past few months and although we’re far from done, it’s a good time to review two of these updates:

  1. A few months ago, we started entering information for Hindi films released between 1931 and 1940. In a first pass, we entered the most important films released in this decade – films like “Alam Ara” (1931), “Devdas” (1935) and “Street Singer” (1938). In a second pass, we started filling in information for the remaining films. As of now, we’re done with the filmography for the period 1931 to 1936. Work on the period 1937 and 1940 is currently in progress. We expect this work to be done by April.
  2. MySwar now provides listing of albums by Label and listings of films by Banner/Production house. Labels displayed on album listing pages and on album pages are now hyperlinked. So is the Production company displayed on the album page. One cool thing about these listings are that we have linked related labels and production houses to provide a consolidated listing. For example, Polydor and Music India labels were merged into Universal and so clicking on either of the three gives the same consolidated listing. Same for NFDC and National Film Development Corporation Of India.

Please check out these updates and share any feedback you may have.

Change in MySwar domain

Earlier this week, some of us observed that myswar.com was inaccessible. We had recently moved to a new hosting provider and our first thought was that it was an issue at their end. Further investigation, however, revealed that we were most likely the target of a court order that had ISPs blocking our website. The court order is backed by Section 169A of the IT Act, 2008. We’re not the first website to have been subjected to this arbitrary and draconian law. Websites like Vimeo, Github and Mouthshut have been subjected to such blocks in the past.

One of the biggest issues of such blocks is that the targets have no idea who initiated the block and why. While we will explore legal options to remove the block, with our limited resources, this is going to be extremely difficult.

To work around this issue in the short term, we have moved our domain to http://myswar.co. This comes at great cost to us in terms of our branding and the goodwill of users who’ve encouraged us through the years. Unfortunately, this appears to be our best option for now. The block has also resulted in our apps not working. We’re working to publish the updated versions of the apps by next week.

As experts have noted, copyright owners are increasingly using court orders to mass block torrents and piracy sites around the time major films are released. Unfortunately, this process unfairly sweeps up legal, smaller websites like ours. In an attempt to address this problem, we will defer publishing of film albums to after this period.

We sincerely regret the inconvenience caused to our users and hope that they’ll stay with us while we figure out a long term solution.

MySwar Mobile Web Version Now Available

IMG_3178

Over the last few weeks, we rolled out mobile web updates to the MySwar website. When you access myswar.co from a mobile device, you’ll see an easy-to-navigate, mobile-friendly version. Almost every feature available available on the desktop website is available in the mobile web version. While the UI is different, the flow is very similar to the flow of the desktop website. The mobile website is intuitive but do take time out the check out the feature-rich, context-sensitive Settings option. Depending on what page you are in, the Settings pop-up provides you options to do various things including starting a playlist, filtering lists, logging in, changing the display language and switching to the destop UI. We hope you enjoy this update.

MySwar App Now Available In Hindi

Updated on Mar 12, 2015: The MySwar Android app is also available in Hindi now.

IMG_2018

A little more than a year ago, we had announced the availability of content on MySwar in Hindi in addition to English. We finally got to roll out an update that makes Hindi content available on the MySwar iOS app and the MySwar Android app as well.

Pretty much all content on the app, artist bios and trivia being exceptions, is now available in Hindi. Just go to Settings -> Language -> Choose “हिन्दी में” and Voila! As in the website, regardless of the Language setting in the app, you can search for songs/albums/artists by typing in either English or Devanagari.

Here’s a quick view of how the language setting works:

IMG_2016IMG_2017IMG_2018IMG_2019IMG_2020IMG_2021

Non-Film Albums Now Available On MySwar App

So far we had only Hindi film albums available on the MySwar app. Over the weekend, we rolled out an update that makes non-film albums also available. This means that the best works of artists like Jagjit Singh, Indian Ocean, Shubha Mudgal, Raghu Dixit, Swarathma and Vasuda Sharma are now available on our app. It also means that we get to see another dimension of popular film artists like Lata Mangeshkar, Asha Bhosle, Manna Dey, Mohammed Rafi and Talat Mahmood.

We hope you enjoy this update: Android, iOS.

IMG_1851

 

Film Credits On MySwar

Since MySwar launched 2011, we have steadfastly focused on crediting musicians making Hindi film music – music directors, lyricists, singers and when the information was available, arrangers, assistants, instrumentalists and so on. We believe that musician credits is a sadly overlooked aspect of music metadata in India. That is the reason you didn’t see any credits for the film cast and crew all this time. While we continue to hold that belief, we believe we have made a significant contribution in cataloguing comprehensive and accurate musician credits and it’s now time to start adding other film credits as well.

A few weeks ago we started showing credits for the film crew – specifically Director, Producer, Writer (Story, Dialogue, Screenplay), Cast and Studio. So far we have credits for over 1500 films and you should be able to find complete filmographies of the superstars – Dev Anand, Shammi Kapoor, Rajesh Khanna, Amitabh Bachchan – and the major directors – Anurag Kashyap, Dibakar Banerjee, Gulzar, Guru Dutt, Hrishikesh Mukherjee, Manmohan Desai, Nasir Hussain, Shakti Samanta, Subhash Ghai, Vishal Bhardwaj, Vidhu Vinod Chopra and Yash Chopra, among others. This remains a work in progress effort as we continue to add film credits for other films based on priority decided by the film’s significance and the significance of the film’s cast and crew.

This additional information is available on the website in the album page as well as in the app in the additional information screen for albums. On the website, this information is available in Hindi as well English.

Sholay

One of the challenges we faced in this project is reconciling artists with same or similar names. For example, while Nasir Hussain (नासिर हुसैन) is the producer/director behind films like “Teesri Manzil” (1966), “Yaadon Ki Baarat” (1973) and “Qayamat Se Qayamat Tak” (1988); Nazir Hussain (नज़ीर हुसैन) is the actor known for his role in films like “Devdas” (1955), “Kashmir Ki Kali” (1964), “Jewel Thief” (1967) and “Amar Akbar Anthony” (1977). We have tried our best to ensure proper credits by using the primary source where we could – the credits in the film itself – as well as a number of other sources including the venerable Hindi Film Geet Kosh. If, however, you find mistakes, please do let us know and we’ll fix it.

In addition to regular search and display, you can also use Advanced Search to find songs that include actor/producer/director/writer/studio parameters. The results from these searches are indicative since – a) we don’t have all the films covered yet for these new credit attributes, b) the credits are at the film level, not the song level (relevant specially for actors).

I hope you like this new facet of MySwar and enjoy the delicious nuggets of information it offers.

MySwar App Update On iOS – Going Free!

We released a new version of the MySwar app on iOS yesterday. This update makes the app free and displays banner ads. There is an option to make an in-app purchase to remove the ads for a period of 1 year (Upgrade option in Settings).

We realize that some of you have purchased the app only recently and this move may appear unfair. To address this scenario – we ask you to send us an email at [email protected] with your iOS device UDID and we will help you get back to the ad-free version (for one year). If you need help figuring out the UDID, let us know and we’ll help you.

If you still haven’t downloaded the app, there really is no excuse now. Get it from here: https://itunes.apple.com/app/myswar/id622503117?ls=1&mt=8

The Android app was always free and remains free: https://play.google.com/store/apps/details?id=com.mavrix.myswar

Hindi Version of MySwar – Behind the Scenes

Take a big cup of coffee and settle down comfortably. This is going to take time. I have been part of big projects which include internationalization and localization. But at most places we would have string of parameters which you would translate to the respective language. The app will pickup the required language file and show the UI in respective language. In some cases we would change the way how dates, currencies and floating points were displayed. Myswar was a little different for me. Here we wanted not just the UI but the whole (we still have gaps) content in Hindi. We also wanted search, sort, numeric liberals, dates in Hindi. This blog post will be about that, its meant to be like a reference note to myself for later use and hence in detail.

1. UTF-8 or UTF16? What to use, conversions…

This time before I jumped into developing, I really wanted to understand the meaning of Unicode encodings. Specially what UTF-8 and UTF16 meant, differences etc. The below video explains UTF-8 in a very simple manner

I would suggest you to watch the following detailed videos on ASCII, UTF16/32 and UTF-8 when you get time. It’s very important as a developer to know how characters are represented behind the scene.


UTF8 has won the war in terms of character encoding for the web. Specially because of its backward compatibility with ASCII. So UTF-8 it was for myswar.

2. Translation and Transliteration

We had labels which needed to be translated into Hindi, we used Google Translation API to do that. But as far the album/song titles etc we had to do transliteration. I used my old script to that but the quality of the transliteration wasn’t great. So we kind of had to create our own dictionary words to work along with the script to resolve issues.

3. Application Programming

Application programming is still very difficult when it comes to using local languages. Some issues that we faced while coding of hindi, there could be many more

String length

len(s)

Return the length (the number of items) of an object. The argument may be a sequence (string, tuple or list) or a mapping (dictionary). That’s not actually correct definition. When it comes to strings its the count of bytes by default.

>>> len('म')
3
>>> len('m')
1
>>> len('मt')
4
>>> len('मt'.decode('UTF8'))
2

So you can’t reliably use len to get the count of characters by default. You need to decode it every time. Thats a pain.

Even simple math is impossible

>>> १+९
  File "", line 1
    १+९
    ^
SyntaxError: invalid syntax
>>> 1+9
10

So if you ever stored numbers in Unicode then every time you want to do some kind of math on it you need to translate them into ASCII. Same applies for calendar operations.
Equals don’t always work all the time aka what you see is not what you get

>>> print 'क़' == 'क़'
False
>>> print u'\u0958' == u'\u0915\u093c'
False
>>> import unicodedata
>>> print  unicodedata.normalize('NFC' ,u'\u0958') == unicodedata.normalize('NFC', u'\u0915\u093c')
True
>>> print  unicodedata.normalize('NFC' ,u'क़') == unicodedata.normalize('NFC', u'क़')
True
>>> 

Basically what you see is not what it is inside. The same character (which looks the same) might have a different value. I wont go into details you can read this detailed FAQ on Normalization.

Sorting Unicode
A python list containing Unicode strings wont sort properly. If we want to sort it properly then we need to implement Unicode Collation Algorithm (UCA). You can check the attempt made by James Tauber. Logic is simple and straight forward. I hope it will be built into the python language very soon. Sorting is fixed in most of the new databases. They can mostly sort Unicode columns by default. In mongo its still an issue, they don’t support sorting a column by collation. You need to implement it yourself if you want Unicode sorting.

Search, filter
Since the EQUALS doesn’t work the way you want and regular expression is still very basic and messy. If you have search or filter functionality you will have to do many trial and error. This needs a separate post.

Localization of date and time formats
It’s almost impossible. You will have to make your own routines to display date and time in your own language.

4. Localizing strings in JavaScript

As far as I know there are still no standard ways to implement localization in JavaScript. For example if you have a date object, how do you localize to show the date in Hindi and can we sort it in JavaScript? There are some libraries which help you in direct string replacement but I guess that’s not enough. As of now better idea would be not to depend on JS.

5. Input method – JavaScript – JQuery IME

inputting is still an issue. Many indians don’t have any input software installed. As of now the best way is to have it as part of JavaScript. I found Wikipedia’s jquery.ime very simple to use. I am still experimenting with it.

Each of these issues can be a blog post by themselves. I will write about them in detail in coming days.