Text To Speech, Highlight words as spoken

Technical Support Apryse SDK
1

Hi Everyone,

We’re building a TTS feature into our reader but I’ve encountered strange behavior while highlighting text as it is spoken.

I’m using the follow code to capture the word coordinates and render the highlight as the words are spoken.

for (TextExtractor.Line line = textExtractor.GetFirstLine(); line.IsValid(); line = line.GetNextLine())
{
for (TextExtractor.Word word = line.GetFirstWord(); word.IsValid(); word = word.GetNextWord())
{
var modifiedWord = word.GetString();
var box = word.GetQuad();
…

}
}

AND

var rect = new Rect();
rect.x1 = quads[0];
rect.y1 = quads[1];

rect.x2 = quads[2];
rect.y2 = quads[5];

Highlight = pdftron.PDF.Annots.Highlight.Create(_currentPdfDocument.GetSDFDoc(), Annot.Type.e_Underline, rect);
if (Highlight == null)
return;

Highlight.SetUniqueID(uniqueId);
Highlight.SetColor(new ColorPt(0, 0, 0), 3);

_currentPage.AnnotPushBack(Highlight);

PdfViewer.Update(rect);

As the text is speaking I am able to render most of the highlights. However when zoomed into the page, so that the entire page is not completely visible, it stops rendering my highlights at about 80% of the way down the page. When the entire page is visible, all highlights are rendered correctly. It happens in more than one book. I’ve validated that the coordinates are valid by selecting the highlight and comparing the results with the GetQuad() call above. It just simply stops rendering.

My counterpart working on the iOS client does not experience this issue.

Any thoughts or help you could provide would be very helpful!

2

After many more hours of tweaks I’ve found that changing the last line from

PdfViewer.Update(rect);

to

PdfViewer.Update();

Fixes the issue. That said I have to think that this will have a negative impact on performance! Redrawing everything has to be worse than redrawing a simple rectangle.

3

Hi Justin,

When passing the rect into the PDView.Update method, you must make sure the points are in screen coordinates and not page coordinates. To do so, you can use the following method to convert the point to screen coordinates:
https://www.pdftron.com/api/ios/pdftron/Classes/PTPDFViewCtrl.html#/c:objc(cs)PTPDFViewCtrl(im)ConvPagePtToScreenPt:page_num:

Please let me know if this works for you, and if you have any further questions.

4

Sorry for the delay getting back to you. Your suggestion worked perfectly, thank you for the help!