Overview

We proposed the pipeline in our project proposal seen in Figure 2.

The pipeline was implemented largely as proposed using the Agile methodology, as Epics with self-contained user-stories. This project was essentially done beyond work-hours and on weekend afternoons. As such, we needed a way to break the overall task down into small chunks (that can be executed over lunch for example), while at the same time, have assured code quality when we integrated it all. Having self-contained unit-tests per user-story helped us tremendously in this regard.

Some optimisations were later found necessary for data gathering and processing.

Epic 1: Project Scaffolding

We used the Agile methodology to breakdown the pipeline into a series of epics and JIRA-style user-stories to be implemented sequentially. Each user-story had its own set of self-contained unit-tests. The overall project has a pyproject.toml file declaring all dependencies and a config.yaml to store values for alpha, curated_categories, traversal_depth, quality_filter_threshold, epsilon, cold_start_window_days

Epic 2: Data Acquisition

We combined manual ratings with openly available data from various Wikipedia API’s:

Epic 3: Candidate Pool Construction

The candidate pool was constructed through the following inputs:

1. Seed Articles: The user would suggest a list of 10-15 Wikipedia articles to seed the model first. In the final evaluation, we seeded the following articles:

   Seed Articles for the Final Evaluation

   Caitika 1915 Singapore Mutiny Ngogo chimpanzee war Longest flights Westray to Papa Westray flight Boat Mail Express Shangri-La Tea Horse Road

   List of tallest buildings in India Longest train services of Indian Railways Johor Bahru–Singapore Rapid Transit System Montreal Quebec Montréal–Trudeau International Airport Orlando: A Biography

2. Categories: Querying the full English Wikipedia corpus of over 7 million articles is not just computationally infeasible, but semantically unfocussed. Therefore, we constrained the candidate pool of articles to a curated set of top-level Wikipedia categories specified by the user. This was defined in config.yaml

   We seeded the following categories in the final evaluation:

   Categories for the Final Evaluation

   Transport-related lists of superlatives Military history of Singapore Named passenger trains of India Mahayana sutras Monks of Nalanda Telugu monarchs

   Novels about transgender topics Metafictional novels History of Andhra Pradesh Chinese classic texts Buddhist holy sites in Andhra Pradesh Philosophical skepticism

3. Depth of Category Traversal: The level to which the categories may be traversed can be set in config.yaml We initially tested the model with 1, but executed it with 2.

Epic 4: Feature Engineering

The raw data collected in the previous epic is to be transformed it into model inputs here, through the following features:

Epic 5: Scoring Model

Epic 6: Clustering (Topical Diversity)

Epic 7: Recommendation Through Explore-Exploit

Epic 8: Take Feedback

Evaluation Premise

We measured Mean Absolute Error (MAE) between predicted and actual composite scores. MAE is appropriate here because it treats all errors equally, which is appropriate given that the composite score is bounded [0,1]. For a recommendation system where consistent ranking is more important than avoiding large errors, MAE is perhaps more appropriate than other error metrics such as Root Mean Squared Error (RMSE)

Our assumption was that we would arrive at the following evaluation hierarchy:

• Naive baseline: As mentioned, we used demand gap score directly as a predicted composite score and measure its error against the computed composite score. This is equivalent to setting translator interest weight, .
• Linear Regression: MAE computed from model output and composite score via k-fold cross-validation
• Random Forest: MAE computed against composite score via k-fold cross-validation.

Looking beyond MAE, feature importance scores from Random Forest could be examined to understand which signals drive recommendation quality most strongly.

Recommendation Quality

The model does more than predict scores, it produces a ranked list of recommendations. While we used MAE to tell us how well the model predicts scores, we will need to evaluate whether the ordering of recommendations is correct.

For such an evaluation, we first defined a threshold on the composite score. Given the range [0,1], we took the midpoint, 0.5, as the threshold. Articles above this score will be labelled as good candidates.

We then computed precision as the fraction of recommended articles exceeding this threshold, and compared this across naive baseline, Linear Regression and Random Forest.

Single-user bias

The approach would take a single user’s (the author) preferences (“translator ratings”) and build upon it, which could open up the chance that the translator’s niche interests could diverge from mass readership.

We had earlier proposed exploring this through a simple diagnostic: we compute a correlation between translator ratings and pageview gap across recommended articles. This will lead us to two possible outcomes:

1. High Positive Correlation: The model’s two-sided approach is coherent: the translator’s interests and reader demand are well-aligned.
2. Negative Correlation: Such an outcome would reveal a genuine tension between supply and demand - the translator’s niche interests diverge from mass readership. This is exactly the problem the model is designed to navigate (and therefore, the more interesting finding, academically speaking)

Result 1: Supply Demand Tension

The model’s core motivation - that there could be a genuine tension between a translator’s interests from mass readership - may be validated. Pearson r between a translator’s ratings and demand gap scores across rated articles calculates to -0.282, with a p=0.049.

Result 2: Evaluating Models

Result 3: Evaluating Cold Start Proxy

We computed a Pearson r between the Hindi proxy demand gap and actual Telugu demand gap on a separate set of articles. If r> 0.7, then the proxy would be justified.

We chose a list of generic Wikipedia articles that are likely to be available on both Hindi and Telugu Wikipedias:

At a computed r=0.474, p=0.0040, the choice of a proxy was not justified:

Result 4: GMM Exploit/ Explore Mechanism

We received the following set of recommendations from the GMM model:

The explore/ exploit distinction has partially collapsed here: several explore candidates have composite scores of 1. This is a consequence of what we saw earlier in Results 2: composite score has lost its discriminating power, even as GMM is doing the differentiation work alone.

It is also notable that The World According to Garp has a GMM value of -37.75. This is because it is the only article in the pool whose category, Novels about transgender topics appears in zero rated articles. This makes it genuinely maximally out-of-distribution for the GMM.

That being said, it is quite fascinating to see recommendations emerging from such a diverse set. Seeding Singapore history gave us Robert Brooke-Popham (a British Air Marshal), and the Malay Regiment for exploit and explore from Cluster 2. Likewise, we see Nüwa (a Chinese goddess) and Michel de Montaigne (a French philosopher) for exploit and explore from the same cluster, Cluster 4.

Result 5: Clustering

Clustering resulted an extremely unbalanced set. While Cluster 0 has 53 out of 78 articles (68%), there were 42 disconnected components in 78 articles. This meant the category co-membership graph is very sparse. Effectively, topical diversity was largely undermined, making it one giant cluster and 11 micro-clusters.

This was an entirely unexpected result as we deliberately chose to seed our categories in a diverse manner. Further drilling down, we note that four of twelve curated categories - such as Telugu monarchs or Buddhist holy sites in Andhra Pradesh - contributed zero candidates after quality filtering. This is because English Wikipedia’s coverage of these topics is predominantly at stub or start tier. What is utterly fascinating here is that this is not incidental: the same structural under-representation that motivates Telugu Wikipedia expansion is reproduced on the supply side. The quality filter, which was designed to source translatable articles, inadvertently maps the contours of the original problem.

In effect, our clever multi-armed bandit - the takkari donga - needs to operate in English as well as Telugu. We will not just need to find articles to translate to Telugu, we also would need to find articles a few degrees removed from Telugu culture in English Wikipedia and perhaps work towards improving their quality as well.

Next Steps

The code pipeline exists, and can be generated for any Wikipedia translator across any language pair. An immediate next step could be try the model out with more focussed categories or seed articles, whilst bearing in mind the quality of the articles. Other languages such as Kannada or Tamil could also be used as a cold start proxy.

Alternatively, replacing the language-proxy demand gap with a normalized quality tier score (quality_tier/3.0) provides a content-based signal that is immune to the pageview degeneracy problem. Indeed, doing so gives us the following fallback shortlist:

(Scores of 0.067 reflect quality_tier = 2 normalized to 2/3, while 1 reflects quality_tier = 3)

Conclusion

Whilst such tweaks can address some limitations in the model, to this multi-lingual writer, the problem perhaps is larger and must be seen across languages. The challenge here is not to bring English Wikipedia articles to Telugu Wikipedia, but to bring articles of interest to a Telugu-speaking milieu to both English and Telugu Wikipedias. Takkari donga, the clever multi-armed bandit, was multi-lingual all along. They need to write not just in Telugu, but in that other Indian language spoken and read widely, English.

Same-same but different

But those familiar waters can sometimes be treacherous. The same word could have different meanings in Sanskrit and other languages. False Friends is a Thing, more so in Indic languages than in other language families. I often need to rely on guard-rails such as Sanskrit Dictionary Collections to keep me safe.

Then there are sandhi’s, liasons between words to form compound words. Whilst the individual word may be understandable, the compound word could feel and sound new; you will need to know how the sandhi has been formed in order to untangle the compound word into its constituents.

Same-same not same as sama-sama

Sanskrit has been historically written in range of scripts, down from the earliest Brahmi and Kharosthi scripts, to more contemporary scripts across Asia.

Here’s a good example of how it all intersects. Consider this quote from Kālidāsa mentioned in Wikipedia, written in various Asian scripts:

Sanskrit is written in each of these scripts for liturgical needs in South and South East Asia. The first word is easily understood: it clearly refers to Siva, one of the gods in the Hindu pantheon. The second word is also easily understood: it’s clearly a conjugation of the word raksha or to protect. But wait: how did to protect become bless?

It gets more difficult with the next word, a multi-letter monster in every script mentioned in the graphic. It is a compound word: among others, words meaning ‘language’, ‘god’ and ‘delight’ have been somehow strung together to form a single word. This is where it starts to become complex.

Haksara Wrehastra: Conjunct Letters

For, untangling conjunct words is not just a matter of giving the right amount of spaces. Most Brahmi-originated scripts have conjunct letters, complex rules where letters may be combined in certain combinations to form new letters. 36 consonants can be combined to form 1296 two-letter combinations in Sanskrit, but that is not all. Each of these combinations can be combined with 16 vowels to form even more complex conjuncts. There is no grammatical limit to the number of letters you can combine. At the most extreme, the word कार्त्स्न्य (“entirety”) consists of a five-letter conjunct, र्त्स्न्य (Stiehl 2017).

Whilst I can read many of the scripts listed in Figure 2, I find it easier break conjunct vowels and therefore words into their constituent parts in Telugu script. But that may not be your experience; perhaps you’re used to reading about Ghatōtgacha’s exploits in Balinese script. Which brings us to our first computational challenge: can we transliterate a Sanskrit string into a script of our choice?

1.1 Methods for Transliteration

A naive suggestion could be that an LLM could draft us a quick method in transliterate Telugu strings to Sanskrit. We tried to get Anthropic Claude 2 to do just that:

import { Observable } from 'observable-fns'

const teluguToDevanagari = Observable.create((teluguText) => {

  const mappings = {
    'అ':'अ', 'ఆ':'आ', 'ఇ':'इ', 'ఈ':'ई', 'ఉ':'उ', 
    'ఊ':'ऊ', 'ఋ':'ऋ', 'ఌ':'ऌ', 'ఎ':'ए', 'ఏ':'ऐ', 
    'ఐ':'ऑ', 'ఒ':'ओ', 'ఓ':'औ', 'ఔ':'अउ',
    // ...add remaining mappings
  }

  let devanagariText = ''

  for(let i=0; i<teluguText.length; i++) {
    if(mappings.hasOwnProperty(teluguText[i])) {
      devanagariText += mappings[teluguText[i]]
    } else {
      devanagariText += teluguText[i] 
    }
  }

  return devanagariText

})

export { teluguToDevanagari }

This obviously doesn’t work.

For one, Claude didn’t complete the mappings constant, and instead, provided a lazy // ...add remaining mappings comment, which isn’t helpful. For another, this approach won’t work with conjunct vowels at all, unless you list every single one of the combination there is.

We tried again, this time asking for it to be in Python:

telugu_ranges = [
    ('0C00', '0C7F'), # Telugu
    ('0C80', '0CFF'), # Telugu Supplementary
] 

devanagari_ranges = [
    ('0900', '097F'), # Devanagari
    ('A8E0', 'A8FF'), # Devanagari Extended
]

telugu_to_devanagari_map = {}

for start, end in telugu_ranges:
    for char in range(int(start, base=16), int(end, base=16)+1):
        telugu_char = chr(char)
        devanagari_char = unicodedata.normalize('NFKC', telugu_char)
        if devanagari_char in devanagari_ranges:
            telugu_to_devanagari_map[telugu_char] = devanagari_char

def transliterate(text):
    return ''.join([telugu_to_devanagari_map.get(char, char) for char in text])

Except that… this doesn’t work either. Here’s what happens when we execute the function:

print(transliterate("అందరికీ నమస్కారం"))

అందరికీ నమస్కారం

… which is exactly the same text as the input. I asked Claude what the problem was, and to its credit, it did identify the error: unicodedata.normalize('NFKC') doesn’t convert from the Telugu character-set to Devanagari.

But it didn’t fix the error directly. Only after a few rounds of hallucinations and corrections, did we arrive at this version, where we offset the Telugu character-set by 0x0900 - 0x0C00, which is the difference between the Devanagari and Telugu character-sets:

def transliterate(text):
  
  devanagari_offset = 0x0900 - 0x0C00
  
  output = ""
  for char in text:
    char_code = ord(char)
    
    if 0x0C00 <= char_code <= 0x0CFF:
      dev_code = char_code + devanagari_offset
      output += chr(dev_code)
    else:
      output += char
        
  return output

Let’s run this method against a few examples:

unit_tests = ["అందరికీ నమస్కారం", # "Hello world" in Telugu
                "శివో రక్షతు గీర్వాణభాషారసాస్వాదతత్పరాన్", #The Kalidasa quote above
                "యెలహంక" # See below
            ]
#loop through the unit tests
print(*[transliterate(test) for test in unit_tests], sep='\n')

अंदरिकी नमस्कारं
शिवो रक्षतु गीर्वाणभाषारसास्वादतत्परान्
यॆलहंक

These work! A key point here is that you should not trust output from LLM’s to work blindly; you must test and validate them, not just with unit-tests, but also with real-world examples. We’ll come back to this point again soon.

A linguistic point in our “unit-tests”. The last example there, యెలహంక, is the Telugu spelling of a Bangalore suburb, also written in Kannada as ಯೇಲಹಂಕ. Transliterating Kannada to Telugu is quite straightforward, as was transliterating to Sanskrit (Devanagari) as येलहंक. But if you read the Devanagari version with Hindi pronunciation in mind, you’ll end up reading it as ‘Yelhank’. That’s incorrect; it should be read as ‘Yelahanka’. This is a good example of False Friends among South Asian languages; Hindi and Sanskrit may be written in the same script, Devanagari, but are to be pronounced differently.

But really, this method needs to be generalized for any South (East) Asian script to any other (whilst noting the potential for False Friends). Let’s expand the method to include Unicode ranges for all scripts.

script_unicode_ranges = {
    #north
    'assamese': [(0x0980, 0x09FF)],
    'bengali': [(0x0980, 0x09FF)],
    'devanagari': [(0x0900, 0x097F), (0xA8E0, 0xA8FF)],
    'gujarati': [(0x0A80, 0x0AFF)],
    'gurmukhi': [(0x0A00, 0x0A7F)],  
    'oriya': [(0x0B00, 0x0B7F)],
    'tibetan': [(0x0F00, 0x0FFF)],
    #south
    'tamil': [(0x0B80, 0x0BFF)],
    'telugu': [(0x0C00, 0x0C7F), (0x0C80, 0x0CFF)],
    'kannada': [(0x0C80, 0x0CFF)],
    'malayalam': [(0x0D00, 0x0D7F)],
    'sinhala': [(0x0D80, 0x0DFF)],
    #SE Asia mainland
    'thai': [(0x0E00, 0x0E7F)],
    'lao': [(0x0E80, 0x0EFF)],
    'myanmar': [(0x1000, 0x109F)],
    'khmer': [(0x1780, 0x17FF), (0x19E0, 0x19FF)],
    #SE Asia maritime
    'balinese': [(0x1B00, 0x1B7F)],
    'javanese': [(0xA980, 0xA9DF)],
    'sundanese': [(0x1B80, 0x1BBF)]
}
# write script_unicode_ranges to a file
with open('script_unicode_ranges.json', 'w') as fp:
    json.dump(script_unicode_ranges, fp)

def transliterate(text, source_script, target_script):
  source_range = script_unicode_ranges[source_script]
  target_range = script_unicode_ranges[target_script]

  output = ""
  for char in text:
   
    source_char_code = ord(char)
    
    if in_range(source_char_code, source_range):
       target_char_code = map_to_target(source_char_code, source_range, target_range)
       if target_char_code != 0xFFFD:
        output += chr(target_char_code)
    else:
       output += char
       
  return output

def in_range(char_code, ranges):
    return any([start <= char_code <= end for start, end in ranges])

def map_to_target(source_char_code, source_range, target_range):

  # Calculate target code point
  target_char_code = source_char_code + (target_range[0][0] - source_range[0][0])

  # Check if it is an assigned Unicode character
  if (unicodedata.category(chr(target_char_code)) != 'Cn'):
    return target_char_code 

  # If not, map to unspecified character code point
  return 0xFFFD

Quite a few pieces happening in there. To start with, whilst Brahmi-based scripts are similar to each other, they aren’t same ; for instance, the Bengali script does not have a letter equivalent to the Telugu letter వ, whilst Devanagari and other southern and western scripts do. So we have a check to see if the mapped character is a valid Unicode character; if not, we map it to the Unicode replacement character, 0xFFFD.

Let’s first repeat our “unit-tests”.

#print transliterate for all unit_tests
print(*[transliterate(test, 'telugu', 'devanagari') for test in unit_tests], sep='\n')

अंदरिकी नमस्कारं
शिवो रक्षतु गीर्वाणभाषारसास्वादतत्परान्
यॆलहंक

These work, but let’s try to the Kālidāsa quote with the scripts in Figure 2.

kalidasa_quote = unit_tests[1]
#iterate through keys in script_unicode_ranges
scripts = [script for script in script_unicode_ranges.keys() if script not in ['devanagari', 'telugu']]
table_row_list = [f"{script}| {transliterate(kalidasa_quote, 'telugu', script)}" for script in script_unicode_ranges.keys()]
table_row_text = "|\n".join(table_row_list)
display_markdown(f"| Script | Transliteration |\n| --- | --- |\n{table_row_text}", raw=True)

The approach seems to work for some South Asian scripts, but didn’t for Assamese, Bengali, Tamil, Tibetan or the South East Asian scripts or Tibetan. Some additional characters (zero-width joiners, and vowel diatrical marks, among others) were rendered there. One reason could be because some of the rendered glyphs may not map one-to-one. We will come back to those in a later post. I also have no way of knowing whether it is accurate for Oriya or Sinhala. If you read either, please let me know in the comments if the transliteration is accurate.

1.2 Next Steps: Trying it Out

In a subsequent post, I will share this out via either Observable.js or by somehow getting the Python methods to work on the web (mostly via Gradio, perhaps). We will also investigate further why the method didn’t work for some scripts. For now, consider a simple method that allows us to transliterate text from (some) Brahmi-based scripts to others.

Calling Claude 2

The advent of LLM’s now mean that you can write a quick function in Python that can take in any verse in Sanskrit and return both the prati padārtham and the bhāvam in JSON.

Here’s how.

def generate_sloka_meaning(sloka):
    prompt = f"""
    {HUMAN_PROMPT} Consider the following verse in Sanskrit enclosed in single quotes. Please give a JSON response with the following key/ value pairs:
    1. 'prati padartham': Aword for word translation in English in the form of a markdown table. 
    The table should have the following columns: Word in Devanagari script, Word in Telugu script, Word in Roman Script, English Translation.
    Return the table alone as the value.
    2. 'bhavam': The overall meaning of the verse in English sentences. Return the meaning alone as the value.

    Please give the response in the following JSON format: {'{"prati padartham": "table", "bhavam": "meaning"}'} This is important: do not give anything beyond the JSON response.
    The verse: '{sloka}'
    {AI_PROMPT} Can I check through my answer to ensure I have not missed anything?
    {HUMAN_PROMPT} Yes, please do. Make sure you have not missed anything. Particularly check if you're outputting the right script in the right column. 
    {AI_PROMPT}
            """
    anthropic = Anthropic()
    completion = anthropic.completions.create(
        model="claude-2",
        max_tokens_to_sample=40000,
        prompt=prompt,
    )
    return completion.completion

def extract_prati_padartham(sloka_meaning):
    #extract json value with the key 'prati padartham'
    return json.loads(sloka_meaning)["prati padartham"]

def extract_bhavam(sloka_meaning):
    #extract json value with the key 'bhavam'
    return json.loads(sloka_meaning)["bhavam"]

The function generate_sloka_meaning calls Claude 2 and gets it to generate a prati padartham and bhavam. The former is returned as a Markdown table, while the latter is plain text. We also got Claude 2 to check through the response before returning it to us (per suggestion in Anthropic’s docs). This is because it did not generate the full meaning on some executions.

Let’s now input both verses and see what Claude 2 generates for us.

verses = {
    "11.12": "divi sūryasahasrasya bhaved yugapad utthitā\nyadi bhāḥ sadṛśī sā syād bhāsas tasya mahātmanaḥ",
    "11.32": "śhrī-bhagavān uvācha\nkālo’smi loka-kṣaya-kṛt pravṛddho\nlokan samāhartum iha pravṛttaḥ\nṛte ’pi tvāṁ na bhaviṣyanti sarve\nye ’vasthitāḥ pratyanīkeṣu yodhāḥ",
}

# for each verse, generate the meaning
sloka_meanings = {}
for verse in verses:
    sloka_meanings[verse] = generate_sloka_meaning(verses[verse])

Results

This is what we see for the two verses that Oppenheimer quoted:

( Do note that Claude 2 has gotten a few Roman transliterations wrong. )

1. Light from a Noble or Mighty One

Oppenheimer’s translation focusses on bursting of light from a thousand suns, like in a nuclear explosion, while Claude 2’s translation focuses on thousands of suns rising in the sky (“utthitā”). Oppenheimer translated mahātma as “mighty one”, while the Claude 2 translation renders it as “noble one”.

Which is to say: while in Oppenheimer’s translation, the narrator is in awe of a sudden display of immense power, in the latter, the narrator also ascribes a sense of nobility to a gentle - if less scary - display of power.

2. Death or Time

Oppenheimer translated kālaḥ as “Death”, while the Claude 2 translation rendered it as “Time”. A quick search across multiple Sanskrit dictionaries suggests that kālaḥ can indeed mean both “Death” and “Time”, but the latter is more common.

In particular, Monier Williams 2nd Edition (1899) tried to reconcile both meanings by suggesting that kālaḥ could mean “death by age”, or time as an entity that “destroys all things” (Bharati 2023)

Time has been worshipped as entity in Hinduism since the Vedic Era. Rig Veda Mandala 10, Sukta 190 suggests Time originated first from the Great Ocean in the sky, and only then did the Sun and the Moon come into existence:

Yes, of course, this too needs to be translated with the function we wrote above.

Note that the Claude 2 translation is non-deterministic; each execution has produced a different result.

Time is one of the four primary forms of Vishnu per the Vishnu Purana, the other three being matter (pradhāna), visible substance (vyakta), and spirit (purusha) (via Wikipedia)

In that sense, Krishna telling Arjuna that He is Time is not to say he will destroy the world, but that both creation and destruction are part of the same process, and that Krishna is the source of both.

20th Anniversary Edition (2023)

This is a “20th anniversary edition” of a thesis I wrote in 2002 for the Undergraduate Research Opportunities Programme (UROP), as part of my mathematics minor in the National University of Singapore. This is also a word-for-word republishing of the thesis, but with updates to embed graphics and tables within the text. All errors (if any) are mine, and have existed for 20+ years.

For the longest time, this thesis was up on my then supervisor, Dr Helmer Aslaksen’s website at the Mathematics Department’s webpage. During this time, a great many had downloaded the PDF, but very few presumably understood it. For, this undergrad thesis found it its way to websites talking about dark magic and other related mystical concepts. Probably they got confused with the “tantra” in the title. Didn’t help that I made this mystical-looking, but perfectly scientific, graphic alluding to kaala chakra’s that would presumably sound woo-woo to unsuspecting audiences. Incidentally, I just re-did this graphic (Figure 1) to more exacting design standards, and perhaps would warrant another detailed explanation.

Dr Aslaksen returned to Oslo many many moons ago (pun intended). His old webpage at NUS’ website was still active for many years later, but now that it too has gone, I’m now back on the Internet to say this: nothing tantric about it. I was just punning between the Sanskrit terms ‘panchatantra’ and ‘panchangam’. In fact, this is all mathematic exploration and research, by way of Vedanga Jyotisa, Surya Siddhanta, and modelling all of those principles in Java and LISP. This is me doing that roots thing but via math. This is me claiming expertise in Indic, Chinese and Islamic calendars, of being able to tell when Chinese New Year and Deepavali can fall. Akshay researching kshaya months, if you will. That is all there is to it. Magical, but no black magic.

There is one literary conceit, though.

Second Edition (2002)

The fable of Apara Ganita and the Mystical Garden of Enchanted Numbers is obviously fictional. The inspiration is Leelavati Ganitam, a chapter in the ancient mathematical treatise, the Siddhanta Siromani, written by Bhaskaracharya in 1150 CE. The Leelavati Ganitam is fascinating not only for its treatment of indeterminate analysis and a method to solve Pell’s Equation, but also, as a Canadian university’s website on mathematical history puts it(University 2002), for its poetic conversation between the narrator and a narratee named Leelavati. The similarity between this poetic construct and the conversation between Apara Ganita and the dwara palika is probably noticeable.

Frame stories are not common for scientific research papers, but they certainly have a historical precedent.

2.1 The Southern Amaanta Calendar

The Southern Amaanta Lunisolar Calendar is predominantly followed in the South and South-West Indian states of Andhra Pradesh, Karnataka and Maharashtra. It is essentially a lunisolar one; i.e., its days and months are calculated based on the motions of the moon. Like the Chinese calendar, the month is calculated from new moon to new moon. It however, differs from the Chinese calendar in that the lunar day (“thithi”) of the new moon is considered the last day of the previous month. Again, as in the Chinese calendar, a leap month, an adhika maasa, is added to the calendar every 2.7 years on an average to offset the disparity in lengths between the lunar year and the sidereal year. In addition, a month, the kshaya maasa, is occasionally subtracted. This is discussed in a later segment.

The Southern Amaanta Calendar differs from the Western Amaanta Calendar in its treatment of kshaya maasas, the New Year Day and the Era followed. We believe that the Southern Amaanta Calendar follows the Southern School for treating kshaya maasas. Saha and Lahiri suggest that it follows the Salivahana Saka Era starting with Chaitra Sukla Pratipada (Chakravarty and Chatterjee 1985, 304), the lunar day after the last new moon before Mesha Sankranti. The years are also named according to the names of the Jovian years (Southern School (Saha and Lahiri 1992, 270)). The Eras and handling of kshaya maasas will be discussed in detail in their respective segments.

2.2 Western Amaanta Calendar

As already mentioned, we believe it’s important to distinguish between the Amaanta calendar practised in South and West India. In West India, specifically, in the state of Gujarat, the Amaanta calendar is of two forms(Chakravarty and Chatterjee 1985, 304), one that starts with Aashaadha (followed in the Kathiawar region) and one that starts with Kartika (followed all throughout Gujarat). Both calendars follow the Vikrama Era and both also possibly follow the North Western School for kshaya months.

2.3 Purnimaanta Calendar

The Purnimaanta Calendar is followed in most of North India, i.e., in the states of Bihar, Himachal Pradesh, Uttar Pradesh, Haryana, Punjab, Jammu and Kashmir and Rajasthan (Chatterjee 1988, 42). (Earlier literature fails to mention Uttaranchal, Chattisgarh, Jharkhand and Delhi, but they are off-shots of bigger states and would necessarily follow the same calendar). The last of the three Indian lunisolar calendars, this one differs from the Amaanta calendar in that the months are reckoned from full moon to full moon. Therefore, the Purnimaanta calendar starts two weeks before the Amaanta calendar does; that is, it starts with the lunar day after the last full-moon before Mesha Sankranti. The Vikrama Era is followed (Chakravarty and Chatterjee 1985, 305), along with the Northern School of Jovian-year names (Saha and Lahiri 1992, 270).

2.5 The Malayali Calendar

We now come to the list of Solar Calendars. The Malayali Calendar is predominantly followed in the South Indian state of Kerala. It is essentially a solar calendar; as we shall see later, the months are defined according to the raasis. The year starts with the Simha Sankranti and follows the Kollam Era.(Chakravarty and Chatterjee 1985, 304) The month begins on the same day as a Sankranti if it occurs before aparahna, i.e., three-fifths of a day. Otherwise, it begins on the next day.

2.6 Tamil Calendar

The Tamil calendar is followed in Tamil Nadu. This calendar is also solar; the month begins on the same day as a Sankranti if it occurs before sunset (Chatterjee 1988, 14). The Kali Era is followed along with the Southern Jovian cycle. One peculiarity about the Tamil calendar is that its month names start with Chittirai(Chatterjee 1988, 9) (Chaitra).

2.7 Bengali Calendar

The Bengali calendar is predominantly followed in West Bengal, Assam and Tripura. The Era is the Bengali San. The rule for the beginning of the month is again different; the month begins on the day after a Sankranti, if it occurs before midnight. Otherwise, it begins on the third day. (Chatterjee 1988, 14)

2.8 Oriya Calendar

The Oriya calendar is followed in the Eastern state of Orissa. In addition to the Bengali San, the Saka, Vilayati and Amli eras are followed. (Saha and Lahiri 1992, 258) The month begins on the same day as that of the respective Sankranti. (Chatterjee 1988, 14)

2.9 The Nanakshahi Calendar

Promulgated in 1998 CE, the Nanakshahi Calendar is followed in Punjab. It’s intrinsically linked to the Gregorian calendar, except in its usage of the Nanakshahi Era.(Purewal 2002)

2.10 National Calendar of 1957

Proposed by the Calendar Reform Committee of 1952 and promulgated in 1957 CE, the National Calendar is a tropical calendar with fixed lengths of days and months. However, because it was totally different from the traditional calendars, it did not find much acceptance. (Chatterjee 1988, 18)

We may thus summarize Indian calendars thus:

Do note that: 1. The table is not exhaustive terms of calendars nor in terms of states. Among others, Arunachal Pradesh, Manipur, Meghalaya, Mizoram, Nagaland and Sikkim were left out. 2. Chatterjee mentions that the Orissa School for deciding the beginning of the solar month is also used in Punjab and Haryana “where the solar calendar is also used”. (Chatterjee 1988, 14)

3.1 Months named after Nakshatras

The set of month names named after nakshatras is used by both solar and lunisolar calendars, adding to seeming complexity of the Indian calendar system. Indeed, as we shall see, this type should actually called as ‘Months initially named after Nakshatras’; there has been an infusion of solar rules into an essentially lunar convention.

Let us then, first consider the original rule. Saha and Lahiri mention that pakshas or fortnights were differentiated based on the nakshatra “where the moon is full”.(Saha and Lahiri 1992, 221) That is to say, if a particular full moon occurs near, say, the lunar asterism, Visakha, the full moon would be called as Vaisakha Purnimaasi, and the month would be Vaisakha. The earliest lunisolar months, then, were purnimaanta, that is, the name of the full moon corresponded to the name of the month. Of course, the full moon occurs at all nakshatras. Fifteen were taken into account for naming of the month, spaced more or less equally.

We thus have the following set of names along with their respective nakshatras(Saha and Lahiri 1992, 221):

It may be noted that the months of Aashaadha, Bhadrapada and Phalguna are linked to two nakshatras respectively. Chatterjee and Chakravarthy give the following criteria for choosing nakshatras for month names(Chakravarty and Chatterjee 1985, 281):

1. The yogataaras or the identifying stars of the nakshatras are prominent or have traditional significance.
2. They are spaced equidistant from one another.

It must be mentioned that this rule is now an approximation largely due to Earth’s precession; for instance, this year’s Chitra Purnimaasi had Swati as its nakshatra. Also, possibly for historical reasons, and allowing for regional variation in pronunciation, the Oriya, Bengali, Assamese, Punjabi and Tamil solar calendars also use the same set of month names.

To reconcile all this, we might frame a new rule, that, the amaanta lunar month takes its number from the solar month that starts in it, but its name from the solar month in which it starts, while following the purnimaanta months in chronological order. That is to say, since Chitra occurred during the purnima of this year’s first purnimaanta month, we call this month as ‘Chaitra’. Consequently, the first amaanta month would also be ‘Chaitra’, which also would be the name of the solar month during which the amaanta ‘Chaitra’ started. However, the ‘number’ of the solar month (‘1’ in the case of amaanta and purnimaanta Chaitra) is not quite the same; the solar Chaitra is the last (i.e., 12th) month of the year. The lunisolar Chaitra’s number is taken by the solar month that begins in it, namely the solar Vaisakha. All this can be seen in the graphic in the next page.

The relationships for all the months may be mapped according to the following table (Chakravarty and Chatterjee 1985, 280) (Punjabi month names (Purewal 2002)):

The Assamese, Punjabi and Tamil versions have been provided to give an idea of the linguistic variation. It is also interesting to observe that the National Calendar suggested by Saha and Lahiri also uses the same set of month names, increasing the potential confusion. As is probably obvious by now, the rule does not correspond to the Tamil, National and Nanakshahi calendars.

3.2 Months named after Raasis

Only solar months share their names with raasis. SK Chatterjee and Apurba Kumar Chakravarthy give the following names along with the associated raasis(Chakravarty and Chatterjee 1985, 280):

That is to say, the month shares its name with that of its corresponding Sankranti. For instance, if Mesha Sankranti occurs on a certain day, then the period until the next Sankranti would be Mesha maasa (Medham maasam).

This naming rule is followed primarily in the Malayalam calendar. Incidentally, Abhayankar says that the Oriya calendar also follows this rule. (Abhayankar 1993, 55)

6.1 Treatment of Kshaya Months

We may complete our discussion of kshaya months by describing the three Kshaya Schools of thought. (Chatterjee 1988, 37–40)

The North Western School is followed in the north-western part of the country, presumably in Gujarat and/ or Rajasthan, where the lunisolar calendar is used. Essentially, the North Western School treats the adhika month before kshaya as a normal month and the one after the kshaya month to be intercalary.

This contrasts with the Eastern School where the reverse is followed; the adhika month before the kshaya is deemed intercalary, while the one after it is deemed normal. The Eastern School is followed in the eastern parts of the country, where the lunisolar calendar is followed.

The final of the trio, the Southern School, treats both adhika maasas as intercalary, instead reckoning the kshaya month as a “jugma”, i.e., the first half of the thithi of this month is deemed to be that of the first month, and the second half as that of the second month. This is presumably followed in the Southern parts of the country where the lunisolar calendar is followed.

1. Comments

I have set up comments using Utterances. It’s a free open-source service that uses Github Issues to store comments. You can see the comments on the bottom of this page. Pretty neat, non ?

2. Search

Just love the built-in search in Quatro. Simple but works.

3. Visual Studio Code

Now this is mainly for me, but I don’t have to context-switch away from Visual Studio Code just to make a post. In fact, I can even write code in the same document, and it will be executed and the results shown. This also means… I have AI enabled automatically, through GitHub Copilot. Not that it’s helpful, mind you, in fact, for long-form writing, it’s quite annoying. But hey, it exists, and is particularly helpful in all those settings files, and of course, code-chunks.

4. Code Chunks

Need I say more? How cool is this, that I can just write this and it’ll be executed inline?

print("Hello World!")

Hello World!

5. Table of Contents

See the nice bar to the right listing down all the sections? That’s the Table of Contents. It’s automatically generated, and is a nice way to navigate through the document.

6. Internationalisation

తెలుగు, తదితర భాషలలో కూడా సునాయాసంగా టపాయించచ్చు! Je peux facilement écrire en français aussi!

1. RSS Feed

The RSS feed somehow should work, but doesn’t. I’m still trying to figure out why. Fixed it.

2. Posting Comments

You need to have a GitHub account to post comments. I’m not sure if that’s a good thing or a bad thing for you; it certainly is a good thing for me, as I can leverage on GitHub’s plumbing to manage comments. It also acts as a sort of gatekeeper for spam, for now at least.

Also, I really really REALLY wanted Giscus to work, but it doesn’t. Comments on one page show up on all pages. I’m not sure why; presuming it’s a settings thing, but I can’t figure out how to fix it.

Instead, I’m using Utterances, which is a free open-source service that uses Github Issues to store comments. Same-same for you, but different for me in that Giscus uses GitHub Discussions and not Issues. Ah well.

3. No WYSIWYG

There’s no WYSIWYG editor, so you have to write in Markdown. I don’t mind it, but you might.

4. Executing Python code-chunks

Oh boy, this one was a doozy.

So the main attraction for Quarto/ QMD is that you can write code-chunks in any language, and it’ll be executed and the results shown. However, in Python, I typically create a new conda virtual environment for each project, with a limited set of packages for that project. Now, in the Visual Studio Code - Jupyter notebook world, this is crystal-clear; there’s a convenient dropdown to select the environment. But in Quarto, it’s not so clear.

This is made even harder by the fact that the Quarto plug-in for Visual Studio Code pops up an interactive window when you execute a code-chunk. And that window somehow does have a dropdown to select the environment. But hitting Render on the QMD file doesn’t use that selected environment. Instead, you need to hit command/ctrl-shift-p and select the environment there.

It’s all listed in this issue raised in August 2022. Some helpful checks there too. If you hit an error reading the following:

No module named 'nbformat'

.. then it’s very likely that Quarto is referencing the wrong Python executable. On Macs, this is likely to be the flavour that comes with the OS, not Miniconda/ Anaconda or the environment you just chose. You can check on that by running the following in a code-chunk:

import sys

print(sys.executable)

If the output (in the render) is not the virtual environment you expect, then you know what the problem is. Hit command/ctrl-shift-p and select the right environment.