Be yourself; Everyone else is already taken.
— Oscar Wilde.
This is the first post on my new blog. I’m just getting this new blog going, so stay tuned for more. Subscribe below to get notified when I post new updates.
Introduce Yourself (Example Post)
This is an example post, originally published as part of Blogging University. Enroll in one of our ten programs, and start your blog right.
You’re going to publish a post today. Don’t worry about how your blog looks. Don’t worry if you haven’t given it a name yet, or you’re feeling overwhelmed. Just click the “New Post” button, and tell us why you’re here.
Why do this?
- Because it gives new readers context. What are you about? Why should they read your blog?
- Because it will help you focus you own ideas about your blog and what you’d like to do with it.
The post can be short or long, a personal intro to your life or a bloggy mission statement, a manifesto for the future or a simple outline of your the types of things you hope to publish.
To help you get started, here are a few questions:
- Why are you blogging publicly, rather than keeping a personal journal?
- What topics do you think you’ll write about?
- Who would you love to connect with via your blog?
- If you blog successfully throughout the next year, what would you hope to have accomplished?
You’re not locked into any of this; one of the wonderful things about blogs is how they constantly evolve as we learn, grow, and interact with one another — but it’s good to know where and why you started, and articulating your goals may just give you a few other post ideas.
Can’t think how to get started? Just write the first thing that pops into your head. Anne Lamott, author of a book on writing we love, says that you need to give yourself permission to write a “crappy first draft”. Anne makes a great point — just start writing, and worry about editing it later.
When you’re ready to publish, give your post three to five tags that describe your blog’s focus — writing, photography, fiction, parenting, food, cars, movies, sports, whatever. These tags will help others who care about your topics find you in the Reader. Make sure one of the tags is “zerotohero,” so other new bloggers can find you, too.
What is Regex?
Introduction
Regex or regular expression is a sequence of character that defines a search pattern for a string. Imaging given a string, we need to write a program in Java to differentiate between “A2”, “ABC”, “123”, “-123”, “-123.123”, “123.123” or “1 2 3”. Without regex, it would be hassle implement brute force type of checking that involves looping a character after another and check if it is equal to a matching character. Using regex, we can define our pattern string and check if our string.matches(pattern) .
Rules of Writing Regex
Common Matching Symbol
Defines matching behavior
. Matches any character
^regex Finds regex that must match at the beginning of line
regex$ Finds regex that must match at the end of line
[abc] Set definition, can match a/b/c
[abc][de] Set definition, can match a/b/c followed by d/e
[^abc] Set definition, anything except a/b/c
[a-z0-9] Ranges, matches from a to z or 0 to 9
X|Z Matches X or Z
XZ X followed by Z
$ Check if line end follows
Meta Characters
Defines what the character is
d a digit, short for [0-9]
D a non-digit, short for [^0-9]
s a white space character
S a non-white space character
w a word character
W a non-word character
S+ 1 or more non-white space character
b Matches a word boundary where a word character is[a-zA-Z0-9_]
Quantifier
Defines how often a character should occur
* Occurs zero or more times
+ Occurs 1 or more times
? Occurs either zero or 1 time
{X} Occurs X times
{X,Y} Occurs between X and Y (inclusive)
Grouping
Several characters can be grouped together with a common quantifier using ()
Example Usage
String line1 = "A2";
String pattern1 = "[a-zA-Z][\d]";
String line2 = "ABC";
String pattern2 = "[a-zA-Z]+";
String line3 = "123";
String line4 = "-123";
String line5 = "-123.123";
String line6 = "123.123";
String pattern3 = "-?\d+(\.\d+)?";
String line7 = "3 3";
String pattern4 = "\d\s\d";
String line8 = "dav1d_cheah17@gmail.com";
String line9 = "d2vid-cheah1796@gmail.net";
String pattern5 = "[a-zA-Z0-9]+[\\-]?[\\_]?[a-zA-Z0-9]+@gmail.(com|net)";
(BONUS) Finding the right Regex using Sublime or Atom
At sublime or atom, hit keys command(mac) or control(win) with F to bring up find panel. Select .* which denotes regex and Aa for case sensitive. Simply type the regex expression on the search field and the result will be highlighted.
Conclusion
Regex is powerful that just by using 2 lines, you are able to achieve string matching that was previously done using brute force.
How to do MultiThreading in Java?
What is process and thread?
A process usually refers to a program in execution whereas a thread is part of process. A thread is the smallest part of process (light weight process)that can run concurrently with other threads.
How to create a thread?
Extend Thread class
NOTE: You will not be able to extend to another class after extending to Thread class.
class Runner extends Thread {
public void run() {
for (int i = 0; i < 10; i++) {
System.out.println("Hello " + i);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
public class App {
public static void main(String[] args) {
Runner runner1 = new Runner();
Runner runner2 = new Runner();
runner1.start();
runner2.start();
}
}
Implement Runnable class
class Runner implements Runnable {
public void run() {
for (int i = 0; i < 10; i++) {
System.out.println("Hello " + i);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
public class App {
public static void main(String[] args) {
Thread t1 = new Thread(new Runner());
Thread t2 = new Thread(new Runner());
t1.start();
t2.start();
}
}
Create Thread instance
public class App {
public static void main(String[] args) {
Thread t1 = new Thread(new Runnable() {
public void run() {
for (int i = 0; i < 10; i++) {
System.out.println("Hello " + i);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
});
Thread t2 = new Thread(new Runnable() {
public void run() {
for (int i = 0; i < 10; i++) {
System.out.println("Hello " + i);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
});
t1.start();
t2.start();
}
}
NOTE: Do not use run method as the thread will run in the main thread . Use start instead for it to run on its own thread.
What is Volatile?
Volatile tells a program that the specific variable will not be cached and instead always read and write to main memory. This solves the issue that the variable in CPU cache might not have the correct value as the one in main memory when there is multiple thread accessing to that variable.
Synchronized method vs code blocks(with intrinsic locks)?
- Methods marked with
synchronizedensures no more than 1 invocation ofsynchronizedmethod of same object at the same time. When one thread is executing asynchronizedmethod for an object, all other threads that invoke thesynchronizedmethods of the same object are blocked until the first thread is done with the object. Object in this case is the class.
package demo;
public class App {
private int count;
public static void main(String[] args) throws InterruptedException {
App app1 = new App();
app1.doSth();
}
public synchronized void increment() {
count++;
}
public void doSth() throws InterruptedException {
Thread t1 = new Thread(new Runnable() {
public void run() {
for(int i=0; i<1000; i++) {
increment();
}
}
});
Thread t2 = new Thread(new Runnable() {
public void run() {
for(int i=0; i<1000; i++) {
increment();
}
}
});
t1.start();
t2.start();
t1.join();
t2.join();
System.out.println("Count= " + count);
}
}
- We can have
synchronizedblocks on lock objects within a class to have different threads executing different parts of the code at the same. Hence when thread 1 is busy doing work A, thread 2 can proceed to complete work B at the same time in the same class.
public class App {
private static ArrayList<Integer> list1 = new ArrayList<Integer>();
private static ArrayList<Integer> list2 = new ArrayList<Integer>();
private Object lock1 = new Object();
private Object lock2 = new Object();
private Random random = new Random();
private void task1() {
synchronized(lock1) {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
list1.add(random.nextInt(100));
}
}
private synchronized void task2() {
synchronized(lock2) {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
list2.add(random.nextInt(100));
}
}
private void process() {
for(int i=0; i<1000; i++) {
task1();
task2();
}
}
public static void main(String[] args) {
App app1 = new App();
long start = System.currentTimeMillis();
Thread t1 = new Thread(new Runnable() {
public void run() {
app1.process();
}
});
Thread t2 = new Thread(new Runnable() {
public void run() {
app1.process();
}
});
t1.start();
t2.start();
try {
t1.join();
t2.join();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
long end = System.currentTimeMillis();
System.out.println("Time taken=" + (end - start));
System.out.println("Size of List1=" + list1.size() + " and Size of List2=" + list2.size());
}
}
ThreadPool
Threadpool consists of worker threads. Using worker threads reduces the overhead of creating thread. One common type of thread pool is the fixed thread pool. This type of pool always has a specified number of threads running. The fixed number of working threads ensures the application would not be overloaded and stop working. The application would simply have to wait for threads to complete their task before taking up new ones.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
class Task implements Runnable {
private int id;
public Task(int id) {
this.id = id;
}
public void run() {
System.out.println("Task " + id + " has started.");
try {
Thread.sleep(5000);
} catch (InterruptedException ignored) {
}
System.out.println("Task " + id + " has completed.");
}
}
public class App {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(2);
for(int i=0;i<10;i++) {
executor.submit(new Task(i));
}
executor.shutdown();
System.out.println("All tasks submitted");
long start = System.currentTimeMillis()
;
try {
executor.awaitTermination(1, TimeUnit.DAYS);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
long end = System.currentTimeMillis();
System.out.println("All tasks completed");
System.out.println("Time taken to complete is " + (end-start));
}
}
newFixedThreadPool vs newCachedThreadPool
-
newFixedThreadPoolreuses a fixed number of threads and at any point, at most n Threads will be actively processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available. -
newCachedThreadPoolcreates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available.
Countdown latch
Countdown latch is a threadsafe variable, hence thread synchronization is taken care of. For every invocation of CountDownLatch.countdown the latch count is decremented by one. The CountDownLatch.await temporarily blocks the tread until the count reaches zero.
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
class Task implements Runnable {
private CountDownLatch latch;
private int id;
public Task(CountDownLatch latch, int id) {
this.latch = latch;
this.id = id;
}
public void run() {
try {
System.out.println("Task number " + id + " has started");
Thread.sleep(3000);
} catch (InterruptedException ignored) {
}
latch.countDown();
}
}
public class App {
public static void main(String[] args) {
CountDownLatch latch = new CountDownLatch(5);
ExecutorService executor = Executors.newFixedThreadPool(2);
for(int i=0;i<10;i++) {
executor.submit(new Task(latch, i));
}
executor.shutdown();
System.out.println("All tasks submitted");
try {
latch.await();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("All tasks completed");
}
}
Wait and Notify
Assuming you have 2 threads running. Thread 1 needs completion of thread 2 before resuming. In this case, Thread 1 can use the wait inside synchronized block. The control will be handle over to thread 2 which after completion can call notify inside synchronized block. Handle over from thread 2 to 1 does not happen immediately until the last line in thread 2 synchronized block is run.
public class Task {
public void produce() throws InterruptedException {
synchronized (this) {
System.out.println("Producer thread running ....");
wait();//this.wait() is fine.
System.out.println("Resumed.");
}
}
public void consume() throws InterruptedException {
Scanner scanner = new Scanner(System.in);
Thread.sleep(2000);
synchronized (this) {
System.out.println("Waiting for return key.");
scanner.nextLine();
System.out.println("Return key pressed.");
notify();
Thread.sleep(5000);
System.out.println("Consumption done.");
}
}
}
Reentrant lock
Similar to synchronized method or blocks, when thread 1 acquires a lock, no other thread is able to proceed without thread 1 releasing a lock. The method unlock should always be in finally block incase there is an error and unlock will not be call causing other threads to waiting indefinitely for lock to be released.
import java.util.Scanner;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class Task {
private int count = 0;
private Lock lock = new ReentrantLock();
public void increment() {
for(int i=0;i<10000;i++) {
count++;
}
}
public void firstThread() throws InterruptedException {
lock.lock();
try {
increment();
}
finally {
lock.unlock();
}
}
public void secondThread() throws InterruptedException {
Thread.sleep(2000);
lock.lock();
try {
increment();
}
finally {
lock.unlock();
}
}
public void finishThread() {
System.out.println("The value of count is " + count);
}
}
Wait and Signal
This technique is similar to wait and notify . The difference is after signal is call, the lock is immediately handled over to the thread which call wait . In the wait and notify technique previously described, after notify is called, the control is handled over not until the last line of code in the synchronized block is has executed.
import java.util.Scanner;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class Task {
private int count = 0;
private Lock lock = new ReentrantLock();
private Condition cond = lock.newCondition();
public void increment() {
for(int i=0;i<10000;i++) {
count++;
}
}
public void firstThread() throws InterruptedException {
lock.lock();
System.out.println("Waiting...");
cond.await();
System.out.println("Awaken!");
try {
increment();
}
finally {
lock.unlock();
}
}
public void secondThread() throws InterruptedException {
Thread.sleep(2000);
lock.lock();
System.out.println("Waiting for return key..");
new Scanner(System.in).nextLine();
System.out.println("Has return key");
cond.signal();
try {
increment();
}
finally {
lock.unlock();
}
}
public void finishThread() {
System.out.println("The value of count is " + count);
}
}
Deadlock
Imagine we have an account class, which has getBalance, deposit, withdraw and transfer method. Now we have 2 accounts, and to simulate actual use case, we are going to randomly transfer funds between 2 accounts. This will be done with 2 threads, where first thread randomly transfer from account 1 to 2 and second thread doing the same in opposite way. In the first thread, we will lock account 1 and 2, complete the transfer before releasing both locks. In the second lock, we will do the same except we lock account 2 before 1. This will cause deadlock. At the same time, thread 1 locks account 1 and thread 2 locks account 2. Then thread 1 tries to lock account 2 and thread 2 tries to lock account 1. This will cause both threads waiting for each locks to be release.
import java.util.Random;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class Task {
private Account acc1 = new Account();
private Account acc2 = new Account();
private Lock lock1 = new ReentrantLock();
private Lock lock2 = new ReentrantLock();
public void firstThread() throws InterruptedException {
Random random = new Random();
for (int i = 0; i < 1000; i++) {
lock1.lock();
lock2.lock();
try {
Account.transfer(acc1, acc2, random.nextInt(100));
} finally {
lock1.unlock();
lock2.unlock();
}
}
}
public void secondThread() throws InterruptedException {
Random random = new Random();
for (int i = 0; i < 1000; i++) {
lock2.lock();
lock1.lock();
try {
Account.transfer(acc2, acc1, random.nextInt(100));
} finally {
lock2.unlock();
lock1.unlock();
}
}
}
public void finish() {
System.out.println("Total balance is " + acc1.getBalance() + acc2.getBalance());
}
}
There are 2 ways to solving deadlock. First is always lock both accounts in same order. Second way is using tryLock to lock 2 accounts. This method only acquires a lock if its available and will not block if no lock is available within a time period. Then we will check if both locks have been acquired, else we will release those locks for other threads to use and try again sometime again.
NOTE: During creation of ReentrantLock, you can specify fairness boolean. If it is set to true, lock will be given to longest waiting thread.
package demo;
import java.util.Random;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class Task {
private Account acc1 = new Account();
private Account acc2 = new Account();
private Lock lock1 = new ReentrantLock();
private Lock lock2 = new ReentrantLock();
private void acquireLocks(Lock lock1, Lock lock2) throws InterruptedException {
while(true) {
boolean gotFirstLock = false;
boolean gotSecondLock = false;
try {
gotFirstLock = lock1.tryLock();
gotSecondLock = lock2.tryLock();
} finally {
if (gotFirstLock && gotSecondLock) {
return;
}
if (gotFirstLock) {
lock1.unlock();
}
if (gotSecondLock) {
lock2.unlock();
}
Thread.sleep(1);
}
}
}
public void firstThread() throws InterruptedException {
Random random = new Random();
for (int i = 0; i < 1000; i++) {
acquireLocks(lock1,lock2);
try {
Account.transfer(acc1, acc2, random.nextInt(100));
} finally {
lock1.unlock();
lock2.unlock();
}
}
}
public void secondThread() throws InterruptedException {
Random random = new Random();
for (int i = 0; i < 1000; i++) {
acquireLocks(lock2,lock1);
try {
Account.transfer(acc2, acc1, random.nextInt(100));
} finally {
lock1.unlock();
lock2.unlock();
}
}
}
public void finish() {
System.out.println("First account balance is " + acc1.getBalance());
System.out.println("Second account balance is " + acc2.getBalance());
System.out.println("Total balance is " + (acc1.getBalance() + acc2.getBalance()));
}
}
ReentrantLock vs Synchronized
Some differences between reentrant lock and synchronized blocks are as follows:
- ReentrantLock has fairness attribute which gives the lock to longest waiting thread.
-
tryLockmethod of ReentrantLock only locks its is available within a period of time and reduce thread blocking. -
lockInterruptiblymethod of ReentrantLock allows to interrupt a thread that is waiting for a lock. - ReentrantLock allows listing of threads waiting for a lock.
Semaphore
Semaphore is usually use to restrict the number of threads accessing to a resource a the same time. When we initialize a semaphore, we need to enter the number of permits available. These permits are analogous to number of keys available to access the resource. When a thread calls Semaphore.acquire method, it will try to acquire a permit and if successfully, reduces the number of permits available. When there are no more permits left, other threads would have to wait for permits to be released via Semaphore.release .
Future and Callable
So far we have talked about Runnable which executes a task without returning any value. To return a value from thread we use Callable interface and Future interface. To extract the value returned use Future.get() .
import java.util.Random;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class App {
public static void main(String[] args) throws InterruptedException {
ExecutorService executor = Executors.newCachedThreadPool();
Future<Integer> future = executor.submit(new Callable<Integer>() {
public Integer call() throws Exception {
Random random = new Random();
int duration = random.nextInt(1000);
Thread.sleep(duration);
return duration;
}
});
executor.shutdown();
try {
System.out.println("Duration is " + future.get());
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
NOTE: Both Future and Callable data types must be of same type.
Interrupt Thread
From the main, we can interrupt a thread using Thread.interrupt() and inside a thread, we can catch the interrupt in if(Thread.currentThread().isInterrupted() block.
import java.util.Random;
public class App {
public static void main(String[] args) throws InterruptedException {
System.out.println("Started");
Thread t1 = new Thread(new Runnable() {
@Override
public void run() {
Random ran = new Random();
for (int i = 0; i < 1E8; i++) {
Math.sin(ran.nextDouble());
}
if (Thread.currentThread().isInterrupted()) {
System.out.println("Interrupted");
}
}
});
t1.start();
Thread.sleep(100);
t1.interrupt();
t1.join();
System.out.println("Finished");
}
}
How to work with Java Collections?
Introduction
Collections in Java is a framework that provides architecture to store and manipulate data.
👉 (BONUS) Array-based Coding Interview Question at the end!
Hierarchy
Iterable Interface
This interface is the root interface in Collection framework. It is extended by Collection interface. It contains method:
Iterator<E> iterator() which returns iterator over elements in the list.
Collection Interface
This interface extends Iterable interface. Some of the known subinterfaces include interface List, Queue, Deque, Sets and SortedSets. It contains some of the methods below:
boolean add(Object o)boolean addAll(Object o)void clear()boolean contains(Object o)
Full list of methods supported: https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html
List Interface
This interface is the child interface of Collection and implemented by classes ArrayList, LinkedList, Vector and Stack.
ArrayList Class
This class implements List interface. Some of the properties:
- Can store duplicated element
- Maintains insertion order
- Non-synchronized
- Manipulation is slow because shifting is required if any element is removed from the array
ArrayList vs Array
Array is static because it needs to be initialized with known length. ArrayList is dynamic array and its size increases dynamically as new elements are added. Size of array is obtained using array.length and for arraylist is arraylist.size().
Example of Array Usage:
int arr[] = new int[] {1,2,3};
for (int i=0;i<arr.length;i++) {
System.out.println("Value is " + arr[i]);
}
Example of ArrayList usage:
ArrayList<Integer> arrayList = new ArrayList<Integer>();
list1.add(1);
list1.add(2);
list1.add(3);
// Method1: Fetching elements from Integer ArrayList using for loop
for (int i=0;i<arrayList.size();i++) {
System.out.println("Value is " + arrayList.get(i));
}
// Method2: Fetching elements from Integer ArrayList using iterator
Iterator<Integer> it = arrayList.iterator();
while(it.hasNext()) {
System.out.println("Value is " + it.next());
}
NOTE: To retrieve elements using iterator() , you need to use while loop.
LinkedList Class
This class implements List and Deque interface. It uses double linked list to store element, which means each node contains 3 fields, 2 link fields (references to previous and next element) and 1 data field. Fast for inserting and removing elements but slow in get and set elements.
Some of the properties (similar to that of ArrayList):
- Can store duplicated element
- Maintains insertion order
- Non-synchronized
- Manipulation is fast because no shifting is required
- Can be used as list, stack or queue
Example of usage:
LinkedList<String> linkedList = new LinkedList<String>();
list1.add("test");
list1.add("test");
list1.add("test");
Iterator <String> it = linkedList.iterator();
while(it.hasNext()) {
System.out.println(it.next());
}
ArrayList vs LinkedList
- ArrayList can only act as list whereas LinkedList can act as both list and queue.
- ArrayList is better at storing and accessing data whereas LinkedList is better for manipulating data.
Vector Class
This class implements List interface. It is similar to ArrayList except it is synchronized. Some of the properties:
- Can store duplicated element
- Maintains insertion order
- Synchronized
Example of usage:
Vector<Integer> vector = new Vector<Integer>();
list1.add(1);
list1.add(2);
list1.add(3);
Iterator<Integer> it = vector.iterator();
while(it.hasNext()) {
System.out.println(it.next());
}
Stack Class
This class is subclass of vector. It implements last-in-first-out order of LIFO. Some of the known methods include:
-
Object Push(Object o)—Adding element to the top of stack and return that element -
Object Pop(Object o)— Return and remove element from top of stack -
Object Peek(Object o)— Return element from top of stack without removing it.
Example of usage:
Stack<Integer> stack = new Stack<Integer>();
System.out.println("Push element: " + stack.push(1));
System.out.println("Push element: " + stack.push(2));
System.out.println("Push element: " + stack.push(3));
System.out.println("Pop element: " + stack.pop());
System.out.println("Peek element: " + stack.peek());
Iterator<Integer> it = stack.iterator();
while(it.hasNext()) {
System.out.println(it.next());
}
Result:
Push element: 1
Push element: 2
Push element: 3
Pop element: 3
Peek element: 2
1
2
Queue Interface
This interface is the child interface of Collection and is implemented by class PriorityQueue and extended by Deque. This interface maintains first-in-first-out order or FIFO.
PriorityQueue
This class implements Queue interface. It does not allow null values to be stored in the queue.
Some of the known methods:
-
Object peak()— Retrieves but does not remove the head of the queue, or return null if queue is empty -
Object poll()— Retrieves and removes the head of the queue, or return null if queue is empty
Example of usage:
PriorityQueue<String> queue = new PriorityQueue<String>();
queue.add("test1");
queue.add("test2");
queue.add("test3");
System.out.println("Peek: " + queue.peek());
System.out.println("Poll: " + queue.poll());
Iterator<String> it = queue.iterator();
while(it.hasNext()) {
System.out.println(it.next());
}
Result:
Element: test1
Peek: test1
Poll: test1
test2
test3
Deque Interface
This interface extends Queue interface. Elements can be added or removed from both ends.
ArrayDeque Class
ArrayDeque implements Deque interface. Some of the properties includes not threadsafe and does not permit null elements. This class is likely to be faster than Stack when used as stack and faster than LinkedList when used as queue. Since ArrayDeque implements Deque, it has methods of queue such as peek, pool as well as stack such as push and pop.
Properties include:
- Add or remove elements on both ends
- Do not allow null value
- Not synchronized
- Faster than LinkedList and Stack.
Example usage:
Deque<String> deque = new ArrayDeque<String>();
deque.add("Test1");
deque.add("Test2");
deque.add("Test3");
Iterator<String> it = deque.iterator();
while(it.hasNext()) {
System.out.println(it.next());
}
Set Interface
This interface is a subinterface of Collection. It represents unordered set of elements that does not contain duplicated elements.
HashSet Class
This class implements Set interface. It permits null element. Elements are stored in hash table after being hash. Hence, if there is a duplicated element, the hash value will overwrite the existing in the hashtable. In additional, the elements are stored in unordered structure.
Properties include:
- Stores element by hashing
- Contains unique elements only
- Allows null value
- Not synchronized
- Does not maintain insertion order
Example usage:
HashSet<String> set = new HashSet<String>();
set.add("test1");
set.add("test1");
set.add("test2");
set.add("test2");
set.add("test3");
set.add("test3");
Iterator<String> it = set.iterator();
while(it.hasNext()) {
System.out.println(it.next());
}
Result:
test2
test3
test1
NOTE: The order of insertion is not maintained.
LinkedHashSet
This class implements Set interface. Similar to HashSet, it does not contain duplicated elements and maintains the order of insertion.
Properties include:
- Stores element by hashing
- Contains unique elements only
- Allows null value
- Not synchronized
- Maintains insertion order
Example of usage:
LinkedHashSet<String> set = new LinkedHashSet<String>();
set.add("test1");
set.add("test1");
set.add("test2");
set.add("test2");
set.add("test3");
set.add("test3");
Iterator<String> it = set.iterator();
while(it.hasNext()) {
System.out.println(it.next());
}
Result:
test1
test2
test3
NOTE: The order of insertion is maintained
SortedSet Interface
This is interface extends Set interface and implementing class includes TreeSet class. It provides total ordering on its element.
TreeSet Class
This class implements SortedSet interface in which elements are sorted and does not contain duplicated elements.
Properties include:
- Contains unique elements only
- Does not allow null value
- Not synchronized
- Stores in ascending order
Example of usage.
TreeSet<String> set = new TreeSet<String>();
set.add("orange");
set.add("orange");
set.add("apple");
set.add("apple");
set.add("banana");
set.add("banana");
Iterator<String> it = set.iterator();
while(it.hasNext()) {
System.out.println(it.next());
}
Result:
apple
banana
orange
NOTE: Elements are sorted
(BONUS) Map Interface
The Map interface is not a subtype of Collection interface. However it is part of the Collections Framework. Map allows to store elements in key and value format.
HashMap Class
This class implements Map interface.
Properties include:
- Contains value based on key
- Contains only unique keys
- May have one null key and many null values
- Not synchronized
- Maintains no order
Example usage:
Map<Integer, String>map=new HashMap<Integer, String>();
map.put(1,"test1");
map.put(5,"test2");
map.put(2,"test3");
map.put(6,"test3");
for(Map.Entry <Integer, String> m: map.entrySet()) {
System.out.println(m.getKey()+" "+m.getValue());
}
Result:
1 test1
2 test3
5 test2
6 test3
LinkedHashMap Class
This class extends from HashMap Class. Different from HashMap which does not maintain any order, LinkedHashMap class maintains the insertion order.
Properties include:
- Contains value based on key
- Contains only unique keys
- May have one null key and many null values
- Not synchronized
- Maintains insertion order
Example Usage:
Map<Integer, String>map=new LinkedHashMap<Integer, String>();
map.put(1,"test1");
map.put(5,"test2");
map.put(2,"test3");
map.put(6,"test3");
for(Map.Entry <Integer, String> m: map.entrySet()) {
System.out.println(m.getKey()+" "+m.getValue());
}
Result:
1 test1
5 test2
2 test3
6 test3
TreeMap Class
This class implements SortedMap interface.
Properties include:
- Contains value based on key
- Contains only unique keys
- Cannot have a null key and many null values
- Not synchronized
- Maintains ascending order by key
Example Usage
TreeMap<Integer, String>map=new TreeMap<Integer, String>();
map.put(1,"test1");
map.put(5,"test2");
map.put(2,"test3");
map.put(6,"test3");
for(Map.Entry <Integer, String> m: map.entrySet()) {
System.out.println(m.getKey()+" "+m.getValue());
}
Results:
1 test1
2 test3
5 test2
6 test3
(BONUS) HashTable Class
This legacy class inherits Dictionary class and implements Map interface. Similar to HashMap, both stores data in the form of key and value and uses hashing to store unique key. However some differences are:
- HashTable is synchronized
- HashTable does not allow null key or value
- HashTable is slow
(BONUS) Array-Based Coding Interview Questions
How to find missing number in integer array?
The secret is using an algorithm to get the sum of series highlighted in bold below
public class TestFindMIssingNumberInArray {
public static void main(String[] args) {
printMissingNumber(new int[] {1,2,3,5,6}, 6);
}
private static void printMissingNumber(int[] input, int totalCount) {
int expectedSum = totalCount * (totalCount+1)/2;
int actualSum = 0;
for(int i=0;i<input.length;i++) {
actualSum += input[i];
}
System.out.println("Missing number is " + (expectedSum - actualSum));
}
}
How to find duplicated numbers in integer array?
There are 2 solutions to it. If the question does not allow solving using java.util.* packages, then refer to removeDuplicateBruteForce. The removeDuplicateCollection involves utilizing the properties of Set interface which does not allow duplicates. Use TreeSet to display result in sorted and ascending order.
import java.util.Arrays;
import java.util.Iterator;
import java.util.TreeSet;
public class TestFIndDuplicatesInArray {
public static void main(String[] args) {
removeDuplicateBruteForce(new int[] {1,1,1,2,2,2,3,3,3,3}); //sorted
removeDuplicateCollection(new int[] {1,2,3,4,6,5,7,7,9,8}); //unsorted
}
public static void removeDuplicateBruteForce(int[] input) {
Arrays.sort(input);
int[] result = new int[input.length];
int duplicateCount = 0;
int previous = input[0];
result[0] = previous;
for(int i=0;i<input.length;i++) {
if(input[i] != previous) {
result[i] = input[i];
previous = result[i];
} else {
duplicateCount++;
}
}
int count = 0;
int[] finalResult = new int[input.length - duplicateCount + 1];
for(int i=0;i<result.length;i++) {
if(result[i] != 0) {
finalResult[count] = result[i];
count++;
}
}
System.out.println(Arrays.toString(finalResult));
}
public static void removeDuplicateCollection(int[] input) {
TreeSet<Integer> set = new TreeSet<Integer>();
for(int i: input) {
set.add(i);
}
int[] output = new int[set.size()];
int count = 0;
Iterator<Integer> it = set.iterator();
while(it.hasNext()) {
output[count] = it.next();
count++;
}
System.out.println(Arrays.toString(output));
}
}
How to find all pairs of an integer array who sum is equal to a given number?
There are 2 ways of solving it. One is using brute force and the other is using HashSet or LinkedHashSet. Using Set interface has advantages that any duplicates are remove and we get to use Collection method of contains() to quickly find if the set contains the number that we want. It is equivalent to a for loop block but only with 1 line of code.
package com.example;
import java.util.HashSet;
public class TestSumPairArray {
public static void main(String[] args) {
pairSumBruteForce(new int[] {2, 4, 7, 5, 9, 10, -1}, 9);
System.out.println(".....");
pairSumHashSet(new int[] {2, 4, 7, 5, 9, 10, -1}, 9);
}
public static void pairSumBruteForce(int[] number, int sum) {
for(int i=0;i<number.length;i++) {
int first = number[i];
for(int j=i+1;j<number.length;j++) {
int second = number[j];
if(first+second==sum) {
System.out.printf("(%d,%d) %n", first, second);
}
}
}
}
public static void pairSumHashSet(int[] number, int sum) {
HashSet<Integer> set = new HashSet<Integer>();
for(int value: number) {
int target = sum - value;
if(!set.contains(target)) {
set.add(value);
} else {
System.out.printf("(%d,%d) %n", value, target);
}
}
}
}
How to work with Servlet, JSP and JDBC?
- A servlet is a Java class that can be loaded dynamically into and run by special web server called the servlet container — most popular one is Tomcat.
- Servlets interact with clients via request-response model based on HTTP (HTTPS also supported).
- A servlet can serve static content such as HTML pages and images file, but it would be faster if those contents are served by a HTTP server such as Apache web server or Microsoft Internet Information Server.
- A servlet is loaded by servlet container the first time it is requested. The servlet then is forward the user request, process it and returns the response to the servlet container which then sends the response back to user.
- Using Deployment Descriptor (web.xml), the servlet container will know which servlet to call upon receiving a request from client. Alternatively on servlet 3.0 onwards, we can use annotations such as
@WebServlet(“/abc.html”)to map servlets.
Setup Eclipse IDE for Java EE
- Download Eclipse from this link — https://www.eclipse.org/downloads/packages/release/mars/r/eclipse-ide-java-ee-developers
- Extract the downloaded item to your computer. In the folder, launch eclipse application (no installation needed).
- When the application successfully launches, make sure you’re in the Java EE perspective. You can switch to this perspective by click “Open Perspective” icon located on the top right and select “Java EE”.
Download Tomcat and add to Eclipse
- Go to this link and it will show you which Tomcat version you should be interested to work with — https://tomcat.apache.org/whichversion.html
- Download the interested Tomcat version and extract to your computer.
- Go back to Eclipse. At the “Server” tab in the bottom pane, click the link to add a new server. Select the Tomcat version that you have downloaded previously. At “Server runtime environment”, click add and paste the path containing the extracted Tomcat version folder. Click finish.
- Right click on your added Tomcat server and click start. The server should start successfully.
- You can configure your Tomcat server for example HTTP/1.1 port number by double clicking the Tomcat server to bring up the overview window.
Start creating web project
- Right click at Project Explorer and create new “Dynamic Web Project”. Enter project name and click finish.
- Right click at your project in the Project Explorer and create new HTML file of name index.html in default location of WebContent.
- In the index.html, let’s create a login form within the body tag to takes in username and password and simply display the username upon form submit.
<!DOCTYPE html>
<html>
<body>
<form action=login method="post">
Please enter username and password:<br>
Username: <input type="text" name="username"><br>
Password <input type="text" name="password"><br>
<input type="submit">
</form>
</body>
</html>
4. Run the program by right clicking at the your project name and select “Run As” and “Run on Server”. In this step, the first page will be the index.html.
5. However if you click run button and your active window is on another page (for example, page2.html) then it will be shown in the browser instead of the index.html. For the next following examples, we will be run our application at our LoginServlet.java.
6. Inside /Java Resources/src, let’s create a java class called LoginServlet. This class will extends HttpServlet class. Since we are running our application directly from LoginServlet (meaning we are not using index.html), we need to build our form inside our servlet. For this, we will create a method to show our login form.
public void showLoginForm(HttpServletRequest req, HttpServletResponse res) throws IOException {
res.setContentType("text/html");
PrintWriter out = res.getWriter();
out.println("<html>");
out.println("<head>");
out.println("<title>Login</title>");
out.println("</head>");
out.println("<body>");
out.println("<br>Please enter username and password");
out.println("<form method=post>");
out.println("<br>Username: <input type=text name=username>");
out.println("<br>Password: <input type=text name=password>");
out.println("<br><input type=submit>");
out.println("</form>");
out.println("</body>");
out.println("</html>");
}
HTTP methods — doGet and doPost
- Let’s extend our LoginServlet class to HttpServlet. This extension allows us to use the method
doGetanddoPost. Get and Post are common methods of the HTTP to indicate the kind of action that we want. Here are definition of some of the methods commonly used:
- DELETE — Removes specified resource
- GET — Fetches data from server
- POST — Sends data to server
- PUT — Replaces current representation of target resource with uploaded content
2. Since we want to display our form to the client browser is equivalent to GET method, we should show our form in the doGet method. When the user clicks submit, is equivalent to POST method in which we send data back to server. Hence in the doPost method, we retrieve the data using getParameter method.
package com.davidcheah;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.RequestDispatcher;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
@WebServlet("/login")
public class LoginServlet extends HttpServlet {
public void showLoginForm(HttpServletRequest req, HttpServletResponse res) throws IOException {
res.setContentType("text/html");
PrintWriter out = res.getWriter();
out.println("<html>");
out.println("<head>");
out.println("<title>Login</title>");
out.println("</head>");
out.println("<body>");
out.println("<br>Please enter username and password");
out.println("<form method=post>");
out.println("<br>Username: <input type=text name=username>");
out.println("<br>Password: <input type=text name=password>");
out.println("<br><input type=submit>");
out.println("</form>");
out.println("</body>");
out.println("</html>");
}
public void doGet(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
showLoginForm(req, res);
}
public void doPost(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
String username = req.getParameter("username");
String password = req.getParameter("password");
if (username.isEmpty() || password.isEmpty()) {
showLoginForm(req, res);
} else {
out.println("Hello: " + username);
}
}
}
Deployment Description (web.xml)
- In the index.html form, when we click on submit button and ask for
/loginpath, we really want to callLoginServletbut our application does not know to do that. Hence we need to link both via deployment description of web.xml.
2. In the web.xml, for every servlet we need to add 2 tags (servlet and servlet-mapping) with 2 sub-tags each:
<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://xmlns.jcp.org/xml/ns/javaee" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd" id="WebApp_ID" version="3.1">
<servlet>
<servlet-name>servlet1</servlet-name>
<servlet-class>com.davidcheah.LoginServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>servlet1</servlet-name>
<url-pattern>/login</url-pattern>
</servlet-mapping>
</web-app>
3. The 2 tags, <servlet> and <servlet-mapping> are bound to each other by a common sub-tag <servlet-name> which can be any name. Hence whenever a /login path is requested, from the web.xml it will be able to retrieve to correct servlet class which is LoginServlet (notice you have to provide package name with class name) from the common servlet name tag.
4. Restart the server and you should see the application working.
Web Annotation
- Another way to link servlet class to url is to use web annotation. In the LoginServlet class, just before the class name, add the following:
@WebServlet("/login")
Request Dispatcher
- Let’s create the second servlet that will display welcome message to user.
- Don’t forget to create the 2 tags in web.xml for every new servlet created.
<servlet>
<servlet-name>servlet2</servlet-name>
<servlet-class>com.davidcheah.WelcomeServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>servlet2</servlet-name>
<url-pattern>/welcome</url-pattern>
</servlet-mapping>
3. In the LoginServlet class, we can call another servlet using RequestDispatcher or redirection method. Using the RequestDispatcher method, we can pass information to request to be retrieve by second servlet using setAttribute method. Finally we use forward to pass the request and response to second servlet
package com.davidcheah;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.RequestDispatcher;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
@WebServlet("/login")
public class LoginServlet extends HttpServlet {
public void showLoginForm(HttpServletRequest req, HttpServletResponse res) throws IOException {
res.setContentType("text/html");
PrintWriter out = res.getWriter();
out.println("<html>");
out.println("<head>");
out.println("<title>Login</title>");
out.println("</head>");
out.println("<body>");
out.println("<br>Please enter username and password");
out.println("<form method=post>");
out.println("<br>Username: <input type=text name=username>");
out.println("<br>Password: <input type=text name=password>");
out.println("<br><input type=submit>");
out.println("</form>");
out.println("</body>");
out.println("</html>");
}
public void doGet(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
showLoginForm(req, res);
}
public void doPost(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
String username = req.getParameter("username");
String password = req.getParameter("password");
if (username.isEmpty() || password.isEmpty()) {
showLoginForm(req, res);
} else {
RequestDispatcher rd = req.getRequestDispatcher("welcome");
req.setAttribute("username", username);
rd.forward(req, res);
}
}
}
4. In the WelcomeServlet, we can retrieve the data passed from previous servlet using getAttribute method. The attribute is type Object , hence we need to have a proper casting. Notice in after request has been dispatched to WelcomeServlet, the url is showing login.
package com.davidcheah;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
public class WelcomeServlet extends HttpServlet{
public void doPost(HttpServletRequest req, HttpServletResponse res) throws IOException {
String username = (String) req.getAttribute("username");
PrintWriter out = res.getWriter();
out.println("Hello: " + username);
}
}
Send Redirect
- So let’s recap, a servlet can call another servlet using
forwardwithout responding back to client.sendRedirectinvolves servlet responding back to client and asking to go to another url that leads to second servlet. Meaning there is a round trip back to the client. - Using
sendRedirect, you can pass information between first servlet to next request by appending query the destination url and usingreq.getParameterto retrieve it in the second servlet. - LoginServlet
package com.davidcheah;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.RequestDispatcher;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
@WebServlet("/login")
public class LoginServlet extends HttpServlet {
public void showLoginForm(HttpServletRequest req, HttpServletResponse res) throws IOException {
res.setContentType("text/html");
PrintWriter out = res.getWriter();
out.println("<html>");
out.println("<head>");
out.println("<title>Login</title>");
out.println("</head>");
out.println("<body>");
out.println("<br>Please enter username and password");
out.println("<form method=post>");
out.println("<br>Username: <input type=text name=username>");
out.println("<br>Password: <input type=text name=password>");
out.println("<br><input type=submit>");
out.println("</form>");
out.println("</body>");
out.println("</html>");
}
public void doGet(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
showLoginForm(req, res);
}
public void doPost(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
String username = req.getParameter("username");
String password = req.getParameter("password");
if (username.isEmpty() || password.isEmpty()) {
showLoginForm(req, res);
} else {
res.sendRedirect("welcome?user=" + username);
}
}
}
4. Since we are passing parameter via url, we need to use doGet to retrieve the information via req.getParameter .
package com.davidcheah;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
public class WelcomeServlet extends HttpServlet{
public void doGet(HttpServletRequest req, HttpServletResponse res) throws IOException {
String username = req.getParameter("username");
PrintWriter out = res.getWriter();
out.println("Hello " + username);
}
}
RequestDispatcher VS sendRedirect Recap
-
RequestDispatcherforwards request to another page without going back to client browser whereassendRedirectgoes back to client browser before requesting new url. - Use req.setAttribute to pass data using
RequestDispatcherand url rewriting usingsendRedirect.
What is JSP?
- We can see to build a form in Server, we have to wrap each line of the HTML with
PrinterWriterand it can be quite a hassle. We can further improve this by using JSP. - JSP which stands for Java Server Pages is an extension of the servlet technology created to support authoring of HTML and XML pages. In MVC model, JSP can be use to represent the view while Servlet is the controller. In JSP, most of the building blocks are in HTML but you can insert Java code blocks whereas for Servlet it is build entirely using Java.
- Let’s create a WelcomePage.jsp to mimic WelcomeServlet.java.
<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Insert title here</title>
</head>
<body bgcolor="blue">
<%
String username = request.getParameter("user");
out.println("Hello user: " + username);
%>
</body>
</html>
4. Back in our LoginServlet.java. We can call our new jsp page using request dispatch or redirection.
public void doPost(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
String username = req.getParameter("username");
String password = req.getParameter("password");
if (username.isEmpty() || password.isEmpty()) {
showLoginForm(req, res);
} else {
res.sendRedirect("WelcomePage.jsp?user="+username);
}
}
5. Let us also extract our login form into a separate jsp file call LoginPage.jsp
<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Login JSP Page</title>
</head>
<body>
<form action=login method="post">
Please enter username and password:<br>
Username: <input type="text" name="username"><br>
Password <input type="text" name="password"><br>
<input type="submit">
</form>
</body>
</html>
6. Previously, we run our application on LoginServlet.java. Since now we have LoginPage.jsp, we can add this as our startup page. To do this, go to web.xml and add the following lines within tag <web-app> . Now, when we run our application at project level, the first startup page will be LoginPage.jsp.
<welcome-file-list>
<welcome-file>LoginPage.jsp</welcome-file>
</welcome-file-list>
7. Since we have extracted our login form, we need to remove method showLoginForm in LoginServlet and redirect user to LoginPage.jsp if there is a login failure. Method doGet can be removed in the servlet since have delegate form display to jsp file.
package com.davidcheah;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.RequestDispatcher;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
@WebServlet("/login")
public class LoginServlet extends HttpServlet {
public void doPost(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
String username = req.getParameter("username");
String password = req.getParameter("password");
if (username.isEmpty() || password.isEmpty()) {
res.sendRedirect("LoginPage.jsp?");
} else {
res.sendRedirect("WelcomePage.jsp?user="+username);
}
}
}
Servlet Life Cycle
- Let’s talk abit about life cycle of servlet. There are 3 lifecycles; init, service and destroy. For each life cycle, there is a dedicated method that will be triggered when servlet goes to specific life cycle.
- The
initmethod is called when the servlet class has been instantiated. You can overwrite this method to write initialization code that needs to run only once, such as loading database driver, initialize value and so on. - The
servicemethod is called whenever servlet needs to respond to a request. - The
destroymethod is called before servlet container removes the servlet instance from service.
JDBC
- JDBC stands for Java Database Connectivity. It is an API that allows to connect and execute query to a database.
- Before using the API, we must download the JDBC driver. For this example, we will be using MySQL as our database.
- Download and extact the jar file from the link — https://dev.mysql.com/downloads/connector/j/
- Copy the jar file and paste it inside Project Name/WebContent/WEB-INF/lib at project explorer window. (I am using mysql-connector-java-5.1.47)
- Let’s add an
initmethod inside our LoginServlet to allow us to load our JDBC database driver. Do take note, on version 5.1 onwards, class name has been changed tocom.mysql.cj.jdbc.Driver.
public void init() {
// 1. Load JDBC driver
try {
Class.forName("com.mysql.cj.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
6. Before we make a connection to our database, we need to of course create a database. At our MySQL Workbench, we create a new database call demowebappdb and select it.
CREATE DATABASE demowebappdb
USE demowebappdb
7. Next, we need to create a new table call users to keep our username and password.
CREATE TABLE users (id INT AUTO_INCREMENT PRIMARY KEY, username VARCHAR(255), password VARCHAR(255))
INSERT INTO users (username, password) VALUES ('john', 'john123')
8. Back to our eclipse, let’s create method call login that will return a boolean TRUE if there is a user in our database with the login name and password. We have loaded our JDBC driver in init method, hence the following steps involve creating a connection, a statement, execute our query and store the response in ResultSet.
9. We need to connect to our database url with database username and password. The format for mysql is “jdbc:mysql://localhost:” + port number (default=3306) + “/” + your database name.
10. Here is our login method. Notice the rs.next() is because the cursor is initially is before the first position. Hence we need to shift a row down to obtain the first result.
public boolean login(String name, String password) {
String url = "jdbc:mysql://localhost:3306/demowebappdb";
String dbUsername = "root";
String dbPassword = "password";
String query = "select id from users where username='" + name + "' and password='" + password + "'";
try {
// 2. Create a connection
Connection con = DriverManager.getConnection(url, dbUsername, dbPassword);
// 3. Create a statement
Statement st = con.createStatement();
// 4. Create a ResultSet
ResultSet rs = st.executeQuery(query);
if (rs.next()) {
// 5. Close all connections
rs.close();
st.close();
con.close();
return true;
}
// 5. Close all connections
rs.close();
st.close();
con.close();
return false;
}
catch (SQLException e) {
System.out.println(e.toString());
} catch (Exception e) {
System.out.println(e.toString());
}
return false;
}
And we are done~
Github
Source code available here: https://github.com/tattwei46/DemoWebAppJava
How to use Python with MySQL
What is Python? Well, it is a high-level and general-purpose programming language that can be used for web development, data science and scripting. According to TIOBE index, Python is ranked top 10 most popular programming language. In this article, I will be sharing how we can use Python to interact with MySQL database.
Python Setup
- Install Anaconda — https://www.anaconda.com/distribution/
- Select python version 3 (Version 2 will stop receiving security updates by 2020)
NOTE: Anaconda is a python and R distribution that aims to provide everything you need:
- core python language,
- python packages,
- IDE/editor — Jupyter and Spyder
- Package manager — Conda, for managing your packages)
MySql Setup
- Download and install MySql Community Server — https://dev.mysql.com/downloads/mysql/
- During the installation setup, you will be prompted for a “root” password in the server configuration step.
- Download and install MySql Workbench — https://dev.mysql.com/downloads/workbench/
- Workbench is GUI tool that allows us to manage your MySql database.
- Launch workbench, at the home page, setup a new connection profile with the configuration (Connection method: Standard (TCP/IP), Hostname: 127.0.0.1,Port:3306,Username: root, Password: yourpassword) and test your connection.
- Double click on your local instance and it should bring you the schemas view where you can see all your databases and tables.
MySql-Connector-Python Setup
- This is the python driver for connecting to MySql server.
- Using terminal and conda to download
conda install -c anaconda mysql-connector-python
Connect to MySql
- Launch Anaconda-Navigator to bring you a list of applications available to install. The application we are interested in is Jupyter Notebook which is a web-based python IDE. Install and launch it.
- In the notebook, create a new python file. In the first cell, write the following code to test the mysql connection.
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password"
)
print(mydb)
3. If successful, you should get an object returned with its memory address
<mysql.connector.connection_cext.CMySQLConnection object at 0x10b4e1320>
Create Database
- Let’s create a database called mydatabase.
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password"
)
mycursor = mydb.cursor()
mycursor.execute("CREATE DATABASE mydatabase")
2. Next, we will try to connect to this new database.
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
Create Table
- In our new database, let’s create a new table called customer to hold information related to your customers.
- In the sql below, we are creating a table with title customers with 3 columns of title id, name and address. Column id stores data of type integer denoted by
INT. Column id is meant to store unique key for each record. This is done using thePRIMARY KEY. It should be also incremented as new record is added byAUTO_INCREMENT. - For column name and address, it stores data of type character of maximum 255, denoted by
VARCHAR(255).
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
mycursor.execute("CREATE TABLE customers (id INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR(255), address VARCHAR(255))")
Add record into table
- Let’s add some data into our new table.
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "INSERT INTO customers (name, address) VALUES (%s, %s)"
val = ("David", "California")
mycursor.execute(sql, val)
mydb.commit()
print(mycursor.rowcount, "record(s) inserted.")
2. To insert multiple data, group the data within square brackets, separated by comma. Replace execute() with executemany().
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "INSERT INTO customers (name, address) VALUES (%s, %s)"
val = [
("Lily", "California"),
("David", "San Francisco"),
("Micheal", "Las Vegas"),
("Sarah", "New York")
]
mycursor.executemany(sql, val)
mydb.commit()
print(mycursor.rowcount, "record(s) inserted.")
Select record from table
- To fetch all data from table, use
SELECT *, whereby asterisks means all.
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "SELECT * FROM customers"
mycursor.execute(sql)
myresult = mycursor.fetchall()
for x in myresult:
print(x)
2. To select specific column, use SELECT <column_title> instead of *. For example SELECT name FROM customers.
3. Instead of fetchall(), we can use fetchone() to retrieve first row of result.
4. We can apply filter to our search using WHERE keyword. NOTE: Notice how the query value is stored in separate variable. This is a technique called parameterized queries to prevent SQL injection.
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "SELECT * FROM customers WHERE address = %s"
adr = ("California", )
mycursor.execute(sql, adr)
myresult = mycursor.fetchall()
for x in myresult:
print(x)
5. We can also limit the number of search result by adding LIMIT.
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
mycursor.execute("SELECT * FROM customers LIMIT 5")
myresult = mycursor.fetchall()
for x in myresult:
print(x)
Sort Records
- We can retrieve data from table in ascending order. To make it in descending order, add
DESCafter name.
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "SELECT * FROM customers WHERE address = %s ORDER by name"
adr = ("California", )
mycursor.execute(sql, adr)
myresult = mycursor.fetchall()
for x in myresult:
print(x)
Delete a record
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "DELETE FROM customers WHERE address = %s"
adr = ("California", )
mycursor.execute(sql, adr)
mydb.commit()
print(mycursor.rowcount, "record(s) deleted.")
Drop a table
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "DROP TABLE customers"
mycursor.execute(sql)
Update Record
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "UPDATE customers SET address = %s WHERE name = %s"
val = ("California", "John", )
mycursor.execute(sql, val)
mydb.commit()
print(mycursor.rowcount, "record(s) updated.")
Join Table
- Let’s create 2 more tables, one is called products storing information such as product_id, name of product and their prices. Another table would be orders, storing all orders with each containing order_id, product_id and customer_id. The reason why we do not bound order_id with product name or customer name is because these values might change in future and to prevent updating value in multiple tables, we usually stick to value that is constant, in this case their index number.
- Create products table
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
mycursor = mydb.cursor()
sql = "CREATE TABLE products (id INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR(255), price VARCHAR(255))"
mycursor.execute(sql)
sql = "INSERT INTO products (name, price) VALUES (%s, %s)"
val = [
("macbook", "2000"),
("iphone", "1000"),
("apple watch", "500")
]
mycursor.executemany(sql, val)
mydb.commit()
print(mycursor.rowcount, "record(s) inserted.")
3. Create orders table
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
mycursor = mydb.cursor()
sql = "CREATE TABLE orders (id INT AUTO_INCREMENT PRIMARY KEY, customer_id INT, product_id INT)"
mycursor.execute(sql)
sql = "INSERT INTO orders (customer_id, product_id) VALUES (%s, %s)"
val = [
("1", "1"),
("1", "2"),
("2", "3"),
("3", "3")
]
mycursor.executemany(sql, val)
mydb.commit()
print(mycursor.rowcount, "record(s) inserted.")
4. Next we would like to join 2 tables. There are 2 types of JOIN query which are the INNER JOIN and OUTER JOIN. The former only shows results in which all criteria of joining are met meaning showing records with common column.
NOTE: JOIN keyword by default means INNER JOIN, hence you can use JOIN or INNER JOIN.
5. We are going to show a list of all customers with their respective order ids. The relationship between customer and their orders are stored in orders table. To do this, we are to show customer name and order id from a joined table of customers and orders with criteria on matching customer id.
NOTE: You can add an alias to each table and reference column name from alias.
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "SELECT
c.name, o.id
FROM customers c
JOIN orders o ON c.id = o.customer_id
"
mycursor.execute(sql)
myresult = mycursor.fetchall()
for x in myresult:
print(x)
6. Below shows the result of using JOIN. Notice the table does not show the full list of customers because some customers do not have any order id.
('Lily', 1)
('Lily', 2)
('David', 3)
('Micheal', 4)
7. To demonstrate OUTER JOIN, this time we are going to join orders to customers table but we are going to SELECT FROM customers instead of orders.
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "SELECT
c.name, o.id
FROM customers c
LEFT JOIN orders o ON c.id = o.customer_id
"
mycursor.execute(sql)
myresult = mycursor.fetchall()
for x in myresult:
print(x)
8. Below shows the result using LEFT JOIN. Notice now we have “Sarah” with no order id in the result. That is the function of LEFT JOIN, meaning to show the full records of the LEFT table (customer) irrespective if the criteria is met. RIGHT JOIN does the same thing except it shows the full records of the right table.
('Lily', 1)
('Lily', 2)
('David', 3)
('Micheal', 4)
('Sarah', None)
9. Now, let’s go back to INNER JOIN but with multiple tables. Since we would like to show a table of orders, containing order id, customer name, product name and product price, our SELECT query should be FROM orders table. Next we would like to join orders with products (by finding matching product id) and customers table(by finding matching customer id).
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="password",
database="mydatabase"
)
mycursor = mydb.cursor()
sql = "SELECT
o.id, c.name, p.name, p.price
FROM orders o
JOIN products p ON o.product_id = p.id
JOIN customers c ON o.customer_id = c.id"
mycursor.execute(sql)
myresult = mycursor.fetchall()
for x in myresult:
print(x)
10. Below are the results from the JOIN query with multiple tables.
(1, 'Lily', 'macbook', '2000')
(2, 'Lily', 'iphone', '1000')
(3, 'David', 'apple watch', '500')
(4, 'Micheal', 'apple watch', '500')
11. Other queries include
- NATURAL JOIN — automatic joining by the database
- CROSS JOIN — list of all possible combinations
- UNION — combination of 2 or more queries.
References
- How SQL injection works — https://tableplus.io/blog/2018/08/sql-injection-attack-explained-with-example.html
- SQL Injection Prevention Cheat Sheet — https://github.com/OWASP/CheatSheetSeries/blob/master/cheatsheets/SQL_Injection_Prevention_Cheat_Sheet.md
Flutter : How to do CRUD with Firebase RTDB
Introduction
In previous post of Flutter : How to do user login with Firebase, we talked about how to implement user login or sign up screen with Firebase authentication. Using the same project, we are going to showcase CRUD or create, read, update and delete operations with Firebase RTDB or real time database in this post.
Getting Started
This project requires you to register your project with Firebase and include the downloaded config file into your project. You can get the steps needed in the previous post mentioned above. This step is only needed if you prefer to setup own Firebase database, else you are free to use mine with the config file I have also included in the github project. Do find the link to the project at the bottom of this post.
👉Step 1: Create model class
So back to where we have left previously, we managed to display Welcome message after user successfully logs into their account. This is shown inside home_page.dart . To keep our to-do application simple, we are just going to store the name of to-do and enable user to mark as completed or not. To store information about each to-do item, we need to have a model class. A model class for a to-do item would look like this:
/models/todo.dart
class Todo {
String key;
String subject;
bool completed;
String userId;
Todo(this.subject, this.userId, this.completed);
Todo.fromSnapshot(DataSnapshot snapshot) :
key = snapshot.key,
userId = snapshot.value["userId"],
subject = snapshot.value["subject"],
completed = snapshot.value["completed"];
toJson() {
return {
"userId": userId,
"subject": subject,
"completed": completed,
};
}
}
Each to-do item is unique and has its own key . Each item has a name or subject , a flag to keep track of its completion or completed and userId of who created this item. To create a new todo, all parameters except key is required to pass into the constructor Todo() . The key is automatically generated by RTDB and stored when new todo is added.
When data is fetched from Firebase RTDB, it is in json format. Hence we have the Todo.fromSnapshot(DataSnapshot snapshot) allows us to map data from json format to Todo format. The toJson() does the opposite which is to map the data back into json format before we upload into Firebase RTDB.
👉Step 2: Initialize query
Back in our home_page.dart , we created a list of todos by using List<Todo> _todoList = new List() . When a list of todos is fetched from Firebase, we will store it into local list variables.
We use final FirebaseDatabase _database = FirebaseDatabase.instance; to get access to the Firebase instance. We then build a query from this instance using :
Query _todoQuery = _database
.reference()
.child("todo")
.orderByChild("userId")
.equalTo(widget.userId);
In this query, using the FirebaseDatabase instance, we retrieve a reference to all the data under path /todo . If you have another level in todo, then your query would be _database.reference().child("todo").child("another level") . Both .orderByChild("xx xx") and .equalTo("xx xx") is what I use to tell Firebase I want a list of todos where each todo’s userId is the one I give to you. Makes sense?
Here is how it looks like in the RTDB:
👉Step 3: Setup listeners
Using the query that we just build above, we are going to attach 2 kinds of stream subscriptions to it. One is onChildAdded and another is onChildChanged . What onChildAdded.listen() does is it listens for any new todo item added into Firebase and receives an event and pass into callback function which in this case is the _onEntryAdded . Same goes for onChildChanged.listen() , which listens for any change of data in the Firebase such as mark todo item as done.
_onTodoAddedSubscription = _todoQuery.onChildAdded.listen(_onEntryAdded);
_onTodoChangedSubscription = _todoQuery.onChildChanged.listen(_onEntryChanged);
So what is the function of the _onEntryAdded ? It catches the event snapshot and converts from json to todo model format and adds to the list of todos.
_onEntryAdded(Event event) {
setState(() {
_todoList.add(Todo.fromSnapshot(event.snapshot));
});
}
For _onEntryChanged function, it retrieves the key from the event snapshot and get the index from the list of todo. Then from the list index, it updates that particular todo with the one from event snapshot.
_onEntryChanged(Event event) {
var oldEntry = _todoList.singleWhere((entry) {
return entry.key == event.snapshot.key;
});
setState(() {
_todoList[_todoList.indexOf(oldEntry)] = Todo.fromSnapshot(event.snapshot);
});
}
To properly unsubscribe to the StreamSubscription , we simply use .cancel() inside the dispose() method
@override
void dispose() {
_onTodoAddedSubscription.cancel();
_onTodoChangedSubscription.cancel();
super.dispose();
}
👉Step 4: Build that ListView
I like to use ListView when needed to iterate over a list of items which is dynamically changing in size and show them in a list. So in this case, we are going to iterate over each todo items in _todoList . ListView takes in itemCount which is simply the size of the todo list, i.e, _todoList.count . ListView also takes in itemBuilder which is the part that will build the single tile to display a single todo item. We are going to use ListTile widget to display a single todo item. ListTile accepts some parameters such as trailing for putting icon or other widget at the right side of ListTile , title and subtitle for display text of 2 different context and sizes, leading similar to trailing but for the left side of the ListTile and others.
On each ListTile , we are going to display a grey tick if the todo is not completed and green tick if the todo is completed. For this, we can use the ternary operator which is ? , similar to an if-else statement.
To use it, we provide a bool check on certain condition (in this case is checking the completed flag for an item in Firebase) and end it with ?
(_todoList[index].completed) ? [Do something if completed] : [Do something if not completed]
Hence, our ListTile looks like this:
child: ListTile(
title: Text(
subject,
style: TextStyle(fontSize: 20.0),
),
trailing: IconButton(
icon: (_todoList[index].completed)
? Icon(
Icons.done_outline,
color: Colors.green,
size: 20.0,
)
: Icon(Icons.done, color: Colors.grey, size: 20.0),
onPressed: () {
_updateTodo(_todoList[index]);
}),
)
And overall ListView :
Widget _showTodoList() {
if (_todoList.length > 0) {
return ListView.builder(
shrinkWrap: true,
itemCount: _todoList.length,
itemBuilder: (BuildContext context, int index) {
String todoId = _todoList[index].key;
String subject = _todoList[index].subject;
bool completed = _todoList[index].completed;
String userId = _todoList[index].userId;
return Dismissible(
key: Key(todoId),
background: Container(color: Colors.red),
onDismissed: (direction) async {
_deleteTodo(todoId, index);
},
child: ListTile(
title: Text(
subject,
style: TextStyle(fontSize: 20.0),
),
trailing: IconButton(
icon: (completed)
? Icon(
Icons.done_outline,
color: Colors.green,
size: 20.0,
)
: Icon(Icons.done, color: Colors.grey, size: 20.0),
onPressed: () {
_updateTodo(_todoList[index]);
}),
),
);
});
} else {
return Center(child: Text("Welcome. Your list is empty",
textAlign: TextAlign.center,
style: TextStyle(fontSize: 30.0),));
}
}
Notice the ListTile is wrapped with another widget call Dismissible . This is a widget that allows user to swipe the entire ListTile to mimic the action swipe to delete.
👉Step 5: Fabulous FAB
Still at the home_page.dart, in the build method that returns a Scaffold , below body , we are going to create a FAB or floating action button. The purpose for this button is to allow user to add new todo into the list. The FAB will show an alert dialog containing a text field for user to input name of new todo.
floatingActionButton: FloatingActionButton(
onPressed: () {
_showDialog(context);
},
tooltip: 'Increment',
child: Icon(Icons.add),
)
For the alert dialog to show up, you cannot just return a AlertDialog and expect it to show. Instead we need to use await showDialog() and return AlertDialog inside this builder. The AlertDialog will house a textfield which its value will be held by a textEditingController with 2 FlatButtons of save and cancel. The save button will obviously get the new todo item name and create a new todo instance before uploading into the Firebase.
_showDialog(BuildContext context) async {
_textEditingController.clear();
await showDialog<String>(
context: context,
builder: (BuildContext context) {
return AlertDialog(
content: new Row(
children: <Widget>[
new Expanded(
child: new TextField(
controller: _textEditingController,
autofocus: true,
decoration: new InputDecoration(
labelText: 'Add new todo',
),
))
],
),
actions: <Widget>[
new FlatButton(
child: const Text('Cancel'),
onPressed: () {
Navigator.pop(context);
}),
new FlatButton(
child: const Text('Save'),
onPressed: () {
_addNewTodo(_textEditingController.text.toString());
Navigator.pop(context);
})
],
);
});
}
👉Step 5: Let’s CRUD
👉Create
For create a new todo item, we will take the name input by user at the TextField inside AlertDialog when they tapped on FloatingActionButton . We instantiate a new todo object with the name input. Finally we upload to Firebase using _database.reference().child(“todo”).push().set(todo.toJson())
_addNewTodo(String todoItem) {
if (todoItem.length > 0) {
Todo todo = new Todo(todoItem.toString(), widget.userId, false);
_database.reference().child("todo").push().set(todo.toJson());
}
}
👉Read
For read, it has been mentioned above that you will need to build a query which is :
_todoQuery = _database
.reference()
.child("todo")
.orderByChild("userId")
.equalTo(widget.userId);
From the query, we will attach 2 listeners which is onChildAdded and onChildChanged which will trigger each respective callback methods with event snapshots. From the event snapshot, we simply convert them into todo class and add to list
_onEntryAdded(Event event) {
setState(() {
_todoList.add(Todo.fromSnapshot(event.snapshot));
});
}
_onEntryChanged(Event event) {
var oldEntry = _todoList.singleWhere((entry) {
return entry.key == event.snapshot.key;
});
setState(() {
_todoList[_todoList.indexOf(oldEntry)] =
Todo.fromSnapshot(event.snapshot);
});
}
To help improve on querying based on userId, it is recommend to set a rule in Firebase RTDB rules. This is call indexing and helps Firebase to optimize your data arrangement improve response time. For more information, refer to Index Your Data by Firebase.
{
/* Visit https://firebase.google.com/docs/database/security to learn more about security rules. */
"rules": {
"todo": {
".indexOn": "userId",
},
".read": true,
".write": true
}
}
👉Update
User can mark each todo item as completed or undo this step. They can simply tap on the tick icon on the right side of each todoListTile . For the update, we need to get the todo.key as we need to access to path /todo/todo-unique-key to be able to update what is in that path. The method is similar to Create in the sense it is using .set() but the difference is the addition of .child(todo.key) in the path.
_updateTodo(Todo todo) {
//Toggle completed
todo.completed = !todo.completed;
if (todo != null) {
_database.reference().child("todo").child(todo.key).set(todo.toJson());
}
}
👉Delete
Deleting item from Firebase is straightforward. Similar to Update, we need to get the correct todo.key but we will use method .remove() .
Do note that there is no listener for item removal unlike listener to item added or changed. Hence there is no way those 2 listeners method will be triggered and get the latest snapshot of the database. For this, we need to manually remove the item from our local
_todoListvariable only when the deletion from Firebase is successful.
_deleteTodo(String todoId, int index) {
_database.reference().child("todo").child(todoId).remove().then((_) {
print("Delete $todoId successful");
setState(() {
_todoList.removeAt(index);
});
});
}
Demo
Here is how the application looks like
Github
Source code available:
https://github.com/tattwei46/flutter_login_demo
Appreciation
Thank you for taking time to read this post. I hope it helps you on your wonderful journey with Flutter. If you find it helpful, please 👏👏👏 to encourage me to write more articles like this 😎
How to use cloud function between Android and Firestore with HTTP
Firebase offers a wide range of solutions that help you easily integrate features such as user authentication and database query into your mobile and web application seamlessly. Although integrating Cloud Firestore directly into your mobile app client may seem hassle-free to interact with database, this may pose security risk as your app could manipulated to gain unauthorised access to your business secret lies inside your app. Integrating Cloud Firestore directly into your mobile app also might cause you to spend more time if you want to expand into web application and need to have the same service. Based on these reasons, it might be a good idea to place your backend services onto the cloud.
Fortunately, Cloud functions allows you to do that. You can deploy backend functions such as database queries, heavy computations and it can be triggered using http request response method.
I have created a simple android app and cloud function script to show you the concept behind it.
Firstly, we will need to install Firebase CLI:
npm install -g firebase-toolsLogin and authenticate yourself via browser using command:
firebase login
Initialise a project and start installing all the dependencies:
firebase init functions
During initialisation, you will be asked to select a new or existing Firebase project. There will be option to select between Javascript or Typescript for your script language and option of enabling ESLint which helps to look for potential issues with your script. For this example, i have selected Javascript and disable ESLint.
After initialisation is done, go to folder functions and you will be writing your backend script in index.js
To deploy the script, simple run:
firebase deploy --only functions
From the terminal, you can see logs of the functions in the script being deployed:
Project Console: https://console.firebase.google.com/project/testfire-6b175/overview
DAVIDs-MBP:functions davidcheah$ firebase deploy --only functions
=== Deploying to 'testfire-6b175'...
i deploying functions
i functions: ensuring necessary APIs are enabled...
✔ functions: all necessary APIs are enabled
i functions: preparing functions directory for uploading...
i functions: packaged functions (42.53 KB) for uploading
✔ functions: functions folder uploaded successfully
i functions: creating Node.js 6 function getAccountInfo(us-central1)...
i functions: updating Node.js 6 function register(us-central1)...
i functions: updating Node.js 6 function login(us-central1)...
i functions: updating Node.js 6 function destroySession(us-central1)...
✔ functions[getAccountInfo(us-central1)]: Successful create operation.
Function URL (getAccountInfo): https://us-central1-testfire-6b175.cloudfunctions.net/getAccountInfo
✔ functions[destroySession(us-central1)]: Successful update operation.
✔ functions[login(us-central1)]: Successful update operation.
✔ functions[register(us-central1)]: Successful update operation.
✔ Deploy complete!
Project Console: https://console.firebase.google.com/project/testfire-6b175/overview
DAVIDs-MBP:functions davidcheah$ firebase deploy --only functions
For this example, user will be shown a login/registration screen. First the user registers into Firestore and the backend script checks for any existing account created.
exports.register = functions.https.onRequest((req, res) => {
if (req.method === 'PUT') {
res.status(403).send('Forbidden!');
return;
}
cors(req, res, () => {
let name = req.query.name;
//validations
if (!name) {
res.status(200).send("Please enter name.");
return;
}
let email = req.query.email;
if (!email) {
res.status(200).send("Please enter email.");
return;
}
let password = req.query.password;
if (!password) {
res.status(200).send("Please enter password.");
return;
}
if (!validator.isLength(password, 3)) {
res.status(200).send("Please enter valid password.");
return;
}
if(!validator.isEmail(email)){
res.status(200).send("Please enter valid email.");
return;
}
//check if user already exists in firestore
var userRef = admin.firestore().collection('users')
var userExists;
userRef.where('email', '==', email).get()
.then(snapshot => {
userExists = snapshot.size;
console.log(`user by email query size ${userExists}`);
//send error if user exists
if(userExists && userExists > 0){
res.status(200).send("Account exists with same email Id.");
return;
}
//add user to database
admin.firestore().collection('users').add({
name: name,
email: email,
password: password
}).then(ref => {
console.log('add user account', ref.id);
res.status(200).send("User account created.");
return;
});
})
.catch(err => {
console.log('error getting user by email', err);
res.status(200).send("System error, please try again.");
});
});
});
Once successful, it will direct user into login screen, where user enters email and password. When the backend checks for a match, it will send a response with unique token string which the app will store in Shared Preference memory. This token is also stored in Firestore.
exports.login = functions.https.onRequest((req, res) => {
if (req.method === 'PUT') {
res.status(403).send('Forbidden!');
return;
}
cors(req, res, () => {
let email = req.query.email;
//validation
if (!email) {
res.status(200).send("Please enter email.");
return;
}
if(!validator.isEmail(email)){
res.status(200).send("Please enter valid email.");
return;
}
let password = req.query.password;
if (!password) {
res.status(200).send("Please enter password.");
return;
}
if (!validator.isLength(password, 3)) {
res.status(200).send("Please enter valid password.");
return;
}
//get password from db and match it with input password
var userRef = admin.firestore().collection('users')
userRef.where('email', '==', email).get()
.then(snapshot => {
if(snapshot.size > 1){
res.status(200).send("Invalid account.");
return;
}
snapshot.forEach(doc => {
console.log(doc.id, '=>', doc.data().name);
var userPass = doc.data().password;
//if password matches, generate token, save it in db and send it
if(userPass && password == userPass){
const tokgenGen = new TokenGenerator(256, TokenGenerator.BASE62);
const tokenStr = tokgenGen.generate();
//save token in db to use for other client request's authentication verification
var tokenData = { email: doc.data().email};
admin.firestore().collection('tokens').doc(tokenStr).set(tokenData);
res.status(200).send("token:"+tokenStr );
}else{
res.status(200).send("Invalid email/password.");
}
});
})
.catch(err => {
console.log('error getting user by email', err);
res.status(200).send("System error, please try again.");
});
});
});
After login is successful, user is directed into account info screen where the balance will be shown. In the background, the app retrieves the unique token string and sends to backend script requesting to get account info. The backend script gets the token and checks for match in Firestore. Once it is a match, user account info will be release.
exports.getAccountInfo = functions.https.onRequest((req, res) => {
if (req.method === 'PUT') {
res.status(403).send('Forbidden!');
return;
}
cors(req, res, () => {
let token = req.query.token;
var tokenData = token;
//validation
if (!token) {
res.status(200).send("Please login");
return;
} else {
var tokenDoc = admin.firestore().collection('tokens').doc(token);
tokenDoc.get()
.then(doc => {
//if token exists then send data otherwise error response
if (!doc.exists) {
console.log('Invalid token');
res.status(200).send("Invalid token");
} else {
console.log('valid token');
// TODO: Add code to get information from db
// Lets assume we get account balance from db
var accountBal = '$200';
res.status(200).send(accountBal);
}
});
}
});
});
Once user is done, logout can be performed to clear any token in the app Shared Preference memory and also in Firestore.
exports.destroySession = functions.https.onRequest((req, res) => {
if (req.method === 'PUT') {
res.status(403).send('Forbidden!');
return;
}
cors(req, res, () => {
let token = req.query.token;
var tokenData = token;
//validation
if (!token) {
res.status(200).send("Please login");
return;
} else {
// Delete token entry from db
var tokenDoc = admin.firestore().collection('tokens').doc(token).delete();
tokenDoc.delete();
res.status(200).send("Delete success");
}
});
});
Here is how it looks like on the app and in Firestore:
Both Android and cloud function source code are available in my github:
https://github.com/tattwei46/cloud_functions_firestore_login
Flutter: How to do CRUD with PostgreSQL? Part 2
I am going to share today some exciting topics such as :
- How to build RESTful api on web server using Dart and Aqueduct with integration of Postgresql.
- How to build Flutter mobile app and perform basic CRUD functions with Aqueduct application. 👈 You are here
From the previous post, I have shared how to setup your web server to implement RESTful api using Aqueduct with Postgresql. In this post, we are going to start building our flutter app to interact with our web application.
How to set up Flutter project
Create a new Flutter project call flutter_crud_demo. If you are using Visual Studio Code, you can create new project by View > Command Pallete > Flutter New Project > Enter project name > Select directory to save your project. When the workspace is done initialising, delete the widget_test.dart and empty the content of the main.dart.
Read function
Future Builder
Future builder allows us to render a list view as soon as we get our list of data asynchronously. Future builder has 2 parameters, one is the future which is the future method we use to retrieve the list of data and the other is builder which is what do we want to build with the data. Using future builder allows us to show to user a CircularProgressIndicator before data is ready and shows ListView when fetching is done. In our scaffold body, replace with the following code.
body: new Container(
child: new FutureBuilder<List<Hero>>(
future: getAll(),
builder: (context, snapshot) {
if (snapshot.hasData) {
return new ListView.builder(
itemCount: snapshot.data.length,
itemBuilder: (context, index) {
return new Column(
crossAxisAlignment: CrossAxisAlignment.start,
children: <Widget>[
new Text(snapshot.data[index].name,
style: new TextStyle(fontWeight: FontWeight.bold)),
new Divider()
]
);
}
);
} else if (snapshot.hasError) {
return new Text("${snapshot.error}");
}
// By default, show a loading spinner
return new CircularProgressIndicator();
},
),
),
Next we need to implement getAll()method to retrieve list of heroes from our web server. Our method returns a Futureof type list of hero. Inside the function, we call and awaithttp.get(_heroesUrl) . Await allows us to wait for a response before proceeding to the next line. The function getAll() needs to be marked with async to be able to use await inside the method. From the response, we retrieve the body message and convert into our
Future<List<Hero>> getAll() async {
final response = await http.get(_heroesUrl);
print(response.body);
List responseJson = json.decode(response.body.toString());
List<Hero> userList = createHeroesList(responseJson);
return userList;
}
We need to import some libraries for some helper functions.
import 'dart:convert';
import 'dart:async';
import 'package:http/http.dart' as http;
Let’s create a class call Hero. This class takes in an integer for id and string as name.
class Hero {
Hero({this.id, this.name});
final int id;
String name;
}
Inside class _MyHomePageState, add static const _heroesUrl to contain our localhost url.
static const _heroesUrl = 'http://localhost:8888/heroes';
Let’s run the Flutter app. Also do remember to start your web server application with aqueduct serve command.
Delete function
From here onwards, we are going to introduce an enum to store the status of any http request. Any http request should return type HttpRequestStatus.
enum HttpRequestStatus {
NOT_DONE,
DONE,
ERROR
}
Dismissible
We are going to implement swipe to delete which is a very common pattern found in mobile apps. To do this, we going to swap Column with Dismissible as shown below. We then request to delete a hero by its id at the onDismissed parameter. The method deleteHero returns a future which is the httpRequestStatus . If the status is done, we will ask to redraw the screen using setState().
return new ListView.builder(
itemCount: snapshot.data.length,
itemBuilder: (context, index) {
var item = snapshot.data[index];
return Dismissible(
key: Key(item.id.toString()),
onDismissed: (direction) async {
httpRequestStatus = await deleteHero(item.id);
if (httpRequestStatus == HttpRequestStatus.DONE) {
setState(() {
snapshot.data.removeAt(index);
});
}
},
background: Container(color: Colors.red),
child: ListTile(title: Text('${item.name}')),
);
});
The method deleteHero is as follows
Future deleteHero(int id) async {
httpRequestStatus = HttpRequestStatus.NOT_DONE;
final url = '$_heroesUrl/$id';
final response = await http.delete(url, headers: _headers);
if (response.statusCode == 200) {
print(response.body.toString());
httpRequestStatus = HttpRequestStatus.DONE;
} else {
httpRequestStatus = HttpRequestStatus.ERROR;
}
return httpRequestStatus;
}
Add function
Let’s add a widget icon under the AppBar to allow user to add hero.
appBar: new AppBar(
title: new Text('Flutter CRUD Demo'),
actions: <Widget>[
IconButton(
icon: Icon(Icons.add),
onPressed: _addHeroService,
tooltip: 'Add New Hero',
)
],
),
In the _addHeroService , we call _openDialogAddHero to push a new screen for user input on hero name. This method returns a new hero name which we will then call createHero with it and if hero name is successfully updated, we will call setState() to redraw the screen.
void _addHeroService() async {
String name = await _openDialogAddHero();
HttpRequestStatus httpRequestStatus = await createHero(name);
if (httpRequestStatus == HttpRequestStatus.DONE) {
setState(() {
});
}
}
Let’s add a static const _headers to store content-type of http header.
static final _headers = {'Content-Type': 'application/json'};
The following is the code to send a create new hero request to web server application.
Future createHero(String name) async {
httpRequestStatus = HttpRequestStatus.NOT_DONE;
final response = await http.post(_heroesUrl,
headers: _headers, body: json.encode({'name': name}));
if (response.statusCode == 200) {
print(response.body.toString());
httpRequestStatus = HttpRequestStatus.DONE;
} else {
httpRequestStatus = HttpRequestStatus.ERROR;
}
return httpRequestStatus;
}
Update function
Finally we are left with one last function which is the ability to update hero name. We want user to tap on existing hero to update the name. For this, we add an onTap inside ListTile which calls method _updateHeroService , passing in id and name of the hero at that index of the list. Similar to the _addHeroService which retrieves name from the pop up dialog and pass into updateHero . The screen is then redrawn if the updateHero is successful.
Future _updateHeroService(int id, String name) async {
String updatedName = await _openDialogUpdateHero(id, name);
HttpRequestStatus httpRequestStatus = await updateHero(id, updatedName);
if (httpRequestStatus == HttpRequestStatus.DONE) {
print(httpRequestStatus.toString());
setState(() {
});
}
}
The following is the code for updateHero
Future updateHero(int id, String name) async {
httpRequestStatus = HttpRequestStatus.NOT_DONE;
final url = '$_heroesUrl/$id';
final response = await http.put(url,
headers: _headers, body: json.encode({'id': id, 'name': name}));
if (response.statusCode == 200) {
print(response.body.toString());
httpRequestStatus = HttpRequestStatus.DONE;
} else {
httpRequestStatus = HttpRequestStatus.ERROR;
}
}
As a recap, we have covered how to build our web server application implementing RESTful api and managed to build a Flutter app to perform basic CRUD functions with PostgreSQL.
Github
https://github.com/tattwei46/flutter_crud_postgresql
If you find this article helpful and educational, do give it some 👏👏👏to encourage me to write more of this in future 🙏
The Flutter Pub is a medium publication to bring you the latest and amazing resources such as articles, videos, codes, podcasts etc. about this great technology to teach you how to build beautiful apps with it. You can find us on Facebook, Twitter, and Medium or learn more about us here. We’d love to connect! And if you are a writer interested in writing for us, then you can do so through these guidelines.
davidcheah's blog 