How to: Highlight Document Syntax

• Oct 29, 2020
• 8 minutes to read

The RichEditControl allows you to create a custom ISyntaxHighlightService implementation to display text in different colors and fonts according to the category of syntax sub-elements. These include keywords, comments, control-flow statements, variables, and other elements. This example describes how to highlight the T-SQL syntax.

Note

The syntax highlight implementation can affect the application’s performance.

Parse Document Into Tokens

A token represents a document range that should be highlighted. You can use third-party libraries or add custom syntax highlight logic to parse a document into tokens and highlight them. You can combine both approaches.

Tip

You can use the DevExpress CodeParser library to parse a document into tokens. Refer to the How to use Syntax Highlight Tokens to implement custom syntax highlighting for a code sample. Note that the library supports limited amount of languages.

Take into account the following requirements when you parse document into tokens:

• Each range in a document should be marked by a token.
• Tokens cannot intersect.
• Tokens are continuous (start after the other token).
• Tokens cannot mark fields, bookmarks or hyperlinks.

Follow the steps below to parse the document into tokens:

1. Call the SubDocument.FindAll method to search for keywords or specific symbols.

   You can use regular expressions to search for a syntax. Specify the DocumentSearchOptions.RegExResultMaxGuaranteedLength property to extend the maximum length of a string that can be obtained in a regular expression search. Access the property via the richEditControl.Options.Search notation.

   XAML

   <dxre:RichEditControl x:Name="richEditControl1" ActiveViewType="Draft">
       <dxre:RichEditControl.SearchOptions>
           <dxre:DXRichEditDocumentSearchOptions RegExResultMaxGuaranteedLength="500"/>
       </dxre:RichEditControl.SearchOptions>
   </dxre:RichEditControl>
   

2. Convert all occurrences to SyntaxHighlightToken objects. You can specify a token’s format options in the object constructor.
3. Check whether the tokens intersect. If not, add them to the tokens collection.
4. Parse the remaining text into tokens and add them to the same collection.
5. Sort the objects in the collection according to their position in the original text.

Show Code

Note

A full code example is available here: How to implement T-SQL language syntax highlighting by creating Syntax Highlight Tokens

CustomSyntaxHighlightService.cs

public class CustomSyntaxHighlightService : ISyntaxHighlightService
{
    readonly Document document;

    Regex _keywords;
    //Declare a regular expression to search text in quotes (including embedded quotes)
    Regex _quotedString = new Regex(@"'([^']|'')*'");

    //Declare a regular expression to search commented text (including multiline)
    Regex _commentedString = new Regex(@"(/\*([^*]|[\r\n]|(\*+([^*/]|[\r\n])))*\*+/)");

    public CustomSyntaxHighlightService(Document document)
    {
        this.document = document;

        //Declare keywords
        string[] keywords = { "INSERT", "SELECT", "CREATE", "TABLE", "USE", "IDENTITY", "ON", "OFF", "NOT", "NULL", "WITH", "SET", "GO", "DECLARE", "EXECUTE", "NVARCHAR", "FROM", "INTO", "VALUES", "WHERE", "AND" };
        this._keywords = new Regex(@"\b(" + string.Join("|", keywords.Select(w => Regex.Escape(w))) + @")\b");
    }

    private List<SyntaxHighlightToken> ParseTokens()
    {
        List<SyntaxHighlightToken> tokens = new List<SyntaxHighlightToken>();
        DocumentRange[] ranges = null;

        // Search for quoted strings
        ranges = document.FindAll(_quotedString);
        for (int i = 0; i < ranges.Length; i++)
        {
            tokens.Add(CreateToken(ranges[i].Start.ToInt(),ranges[i].End.ToInt(), Color.Red));
        }

        //Extract all keywords
        ranges = document.FindAll(_keywords);
        for (int j = 0; j < ranges.Length; j++)
        {
            //Check whether tokens intersect
            if (!IsRangeInTokens(ranges[j], tokens))
                tokens.Add(CreateToken(ranges[j].Start.ToInt(), ranges[j].End.ToInt(), Color.Blue));
        }

        //Find all comments
        ranges = document.FindAll(_commentedString);
        for (int j = 0; j < ranges.Length; j++)
        {
            //Check whether tokens intersect
            if (!IsRangeInTokens(ranges[j], tokens))
                tokens.Add(CreateToken(ranges[j].Start.ToInt(), ranges[j].End.ToInt(), Color.Green));
        }

        // Sort tokens by their start position
        tokens.Sort(new SyntaxHighlightTokenComparer());

        // Fill in gaps in document coverage
        tokens = CombineWithPlainTextTokens(tokens);
        return tokens;
    }

    //Parse the remaining text into tokens:
    List<SyntaxHighlightToken> CombineWithPlainTextTokens(List<SyntaxHighlightToken> tokens)
    {
        List<SyntaxHighlightToken> result = new List<SyntaxHighlightToken>(tokens.Count * 2 + 1);
        int documentStart = this.document.Range.Start.ToInt();
        int documentEnd = this.document.Range.End.ToInt();
        if (tokens.Count == 0)
            result.Add(CreateToken(documentStart, documentEnd, Color.Black));
        else
        {
            SyntaxHighlightToken firstToken = tokens[0];
            if (documentStart < firstToken.Start)
                result.Add(CreateToken(documentStart, firstToken.Start, Color.Black));
            result.Add(firstToken);
            for (int i = 1; i < tokens.Count; i++)
            {
                SyntaxHighlightToken token = tokens[i];
                SyntaxHighlightToken prevToken = tokens[i - 1];
                if (prevToken.End != token.Start)
                    result.Add(CreateToken(prevToken.End, token.Start, Color.Black));
                result.Add(token);
            }
            SyntaxHighlightToken lastToken = tokens[tokens.Count - 1];
            if (documentEnd > lastToken.End)
                result.Add(CreateToken(lastToken.End, documentEnd, Color.Black));
        }
        return result;
    }

    //Check whether tokens intersect
    private bool IsRangeInTokens(DocumentRange range, List<SyntaxHighlightToken> tokens)
    {
        return tokens.Any(t => IsIntersect(range, t));
    }
    bool IsIntersect(DocumentRange range, SyntaxHighlightToken token)
    {
        int start = range.Start.ToInt();
        if (start >= token.Start && start < token.End)
            return true;
        int end = range.End.ToInt() - 1;
        if (end >= token.Start && end < token.End)
            return true;
        if (start < token.Start && end >= token.End)
            return true;
        return false;
    }
}

//Compare token's initial positions to sort them
public class SyntaxHighlightTokenComparer : IComparer<SyntaxHighlightToken>
{
    public int Compare(SyntaxHighlightToken x, SyntaxHighlightToken y)
    {
        return x.Start - y.Start;
    }
}

VB.NET

Public Class CustomSyntaxHighlightService
    Implements ISyntaxHighlightService

    Private ReadOnly document As Document

    Private _keywords As Regex
    ' Declare a regular expression to search text in quotes (including embedded quotes)
    Private _quotedString As New Regex("'([^']|'')*'")

    ' Declare a regular expression to search commented text (including multiline)
    Private _commentedString As New Regex("(/\*([^*]|[\r\n]|(\*+([^*/]|[\r\n])))*\*+/)")

    Public Sub New(ByVal document As Document)
        Me.document = document

        ' Declare keywords
        Dim keywords() As String = { "INSERT", "SELECT", "CREATE", "TABLE", "USE", "IDENTITY", "ON", "OFF", "NOT", "NULL", "WITH", "SET", "GO", "DECLARE", "EXECUTE", "NVARCHAR", "FROM", "INTO", "VALUES", "WHERE", "AND" }
        Me._keywords = New Regex("\b(" & String.Join("|", keywords.Select(Function(w) Regex.Escape(w))) & ")\b")
    End Sub

    Private Function ParseTokens() As List(Of SyntaxHighlightToken)
        Dim tokens As New List(Of SyntaxHighlightToken)()
        Dim ranges() As DocumentRange = Nothing

        ' Search for quoted strings
        ranges = document.FindAll(_quotedString)
        For i As Integer = 0 To ranges.Length - 1
            tokens.Add(CreateToken(ranges(i).Start.ToInt(),ranges(i).End.ToInt(), Color.Red))
        Next i

        'Extract all keywords
        ranges = document.FindAll(_keywords)
        For j As Integer = 0 To ranges.Length - 1

            'Check whether tokens intersect
            If Not IsRangeInTokens(ranges(j), tokens) Then
                tokens.Add(CreateToken(ranges(j).Start.ToInt(), ranges(j).End.ToInt(), Color.Blue))
            End If
        Next j

        'Find all comments
        ranges = document.FindAll(_commentedString)
        For j As Integer = 0 To ranges.Length - 1

            ' Check whether tokens intersect
            If Not IsRangeInTokens(ranges(j), tokens) Then
                tokens.Add(CreateToken(ranges(j).Start.ToInt(), ranges(j).End.ToInt(), Color.Green))
            End If
        Next j

        ' Sort tokens by their start position
        tokens.Sort(New SyntaxHighlightTokenComparer())

        ' Fill in gaps in document coverage
        tokens = CombineWithPlainTextTokens(tokens)
        Return tokens
    End Function

    'Parse the remaining text into tokens:
    Private Function CombineWithPlainTextTokens(ByVal tokens As List(Of SyntaxHighlightToken)) As List(Of SyntaxHighlightToken)
        Dim result As New List(Of SyntaxHighlightToken)(tokens.Count * 2 + 1)
        Dim documentStart As Integer = Me.document.Range.Start.ToInt()
        Dim documentEnd As Integer = Me.document.Range.End.ToInt()
        If tokens.Count = 0 Then
            result.Add(CreateToken(documentStart, documentEnd, Color.Black))
        Else
            Dim firstToken As SyntaxHighlightToken = tokens(0)
            If documentStart < firstToken.Start Then
                result.Add(CreateToken(documentStart, firstToken.Start, Color.Black))
            End If
            result.Add(firstToken)
            For i As Integer = 1 To tokens.Count - 1
                Dim token As SyntaxHighlightToken = tokens(i)
                Dim prevToken As SyntaxHighlightToken = tokens(i - 1)
                If prevToken.End <> token.Start Then
                    result.Add(CreateToken(prevToken.End, token.Start, Color.Black))
                End If
                result.Add(token)
            Next i
            Dim lastToken As SyntaxHighlightToken = tokens(tokens.Count - 1)
            If documentEnd > lastToken.End Then
                result.Add(CreateToken(lastToken.End, documentEnd, Color.Black))
            End If
        End If
        Return result
    End Function

    'Check whether tokens intersect
    Private Function IsRangeInTokens(ByVal range As DocumentRange, ByVal tokens As List(Of SyntaxHighlightToken)) As Boolean
        Return tokens.Any(Function(t) IsIntersect(range, t))
    End Function
    Private Function IsIntersect(ByVal range As DocumentRange, ByVal token As SyntaxHighlightToken) As Boolean
        Dim start As Integer = range.Start.ToInt()
        If start >= token.Start AndAlso start < token.End Then
            Return True
        End If
        Dim [end] As Integer = range.End.ToInt() - 1
        If [end] >= token.Start AndAlso [end] < token.End Then
            Return True
        End If
        If start < token.Start AndAlso [end] >= token.End Then
            Return True
        End If
        Return False
    End Function
End Class

'Compare token's initial positions to sort them
Public Class SyntaxHighlightTokenComparer
    Implements IComparer(Of SyntaxHighlightToken)

    Public Function Compare(ByVal x As SyntaxHighlightToken, ByVal y As SyntaxHighlightToken) As Integer Implements IComparer(Of SyntaxHighlightToken).Compare
        Return x.Start - y.Start
    End Function
End Class

Note

You can use the DocumentRange.Freeze() or DocumentRangeExtensions class methods to improve performance during syntax highlight.

After one of these methods is called, RichEditControl stops tracking the actual document position for this range, and the target ranges cannot be modified. The frozen document ranges become invalid after the document is modified. Don’t use these ranges for further document processing operations.

Specify Token’s Format Options

The SyntaxHighlightProperties class represents a token’s format settings. You can pass this class’s object to the SyntaxHighlightToken object constructor or use it as the SyntaxHighlightToken.Properties property value.

The code sample below converts keywords occurrences to the highlight tokens and specifies their foreground color:

CustomSyntaxHighlightService.cs

SyntaxHighlightToken CreateToken(int start, int end, Color foreColor)
{
    SyntaxHighlightProperties properties = new SyntaxHighlightProperties();
    properties.ForeColor = foreColor;
    return new SyntaxHighlightToken(start, end - start, properties);
}

CustomSyntaxHighlightService.vb

Private Function CreateToken(ByVal start As Integer, ByVal [end] As Integer, ByVal foreColor As Color) As SyntaxHighlightToken
    Dim properties As SyntaxHighlightProperties = New SyntaxHighlightProperties()
    properties.ForeColor = foreColor
    Return New SyntaxHighlightToken(start, [end] - start, properties)
End Function

Apply Syntax Highlight

Note

The RichEditControl highlights unformatted text syntax only.

Call the SubDocument.ApplySyntaxHighlight within the ISyntaxHighlightService.Execute method to enable syntax highlighting.

The code sample below uses the ParseTokens method shown above to create a list of SyntaxHighlightToken objects and pass it to the ApplySyntaxHighlight method.

C#

public void Execute()
{
    List<SyntaxHighlightToken> tSqltokens = ParseTokens();
    document.ApplySyntaxHighlight(tSqltokens);
}
public void ForceExecute()
{
    Execute();
}

VB.NET

Private Sub Execute() Implements ISyntaxHighlightService.Execute
    Dim tSqltokens As List(Of SyntaxHighlightToken) = ParseTokens()
    document.ApplySyntaxHighlight(tSqltokens)
End Sub

Public Sub ForceExecute() Implements ISyntaxHighlightService.ForceExecute
    Execute()
End Sub

In the main class, use the RichEditControl.ReplaceService<T>(T) method to register the created implementation.

C#

public partial class Form1 : Form
{
    public Form1()
    {
        InitializeComponent();

        //Register the created service and load the document
        richEditControl1.ReplaceService<ISyntaxHighlightService>(new CustomSyntaxHighlightService(richEditControl1.Document));
        richEditControl1.LoadDocument("CarsXtraScheduling.sql");

        //Specify the richEdit's layout settings
        richEditControl1.Document.Sections[0].Page.Width = Units.InchesToDocumentsF(80f);
        richEditControl1.Document.DefaultCharacterProperties.FontName = "Courier New";
}

VB.NET

Public Partial Class Form1
    Inherits Form

    Public Sub New()
        InitializeComponent()

       ' Register the created service and load the document
        richEditControl1.ReplaceService(Of ISyntaxHighlightService)(New CustomSyntaxHighlightService(richEditControl1.Document))
        richEditControl1.LoadDocument("CarsXtraScheduling.sql")

        ' Specify the richEdit's layout settings
        richEditControl1.Document.Sections(0).Page.Width = Units.InchesToDocumentsF(80F)
        richEditControl1.Document.DefaultCharacterProperties.FontName = "Courier New"
    End Sub
End Class