Month: November 2022

When it comes to data processing, there are more ways to do it than ever. Your choices include real-time, near real-time, and batch processing. How you do it and the tools you choose depend largely on what your purposes are for processing the data in the first place.

In many cases, you’re processing historical and archived data and time isn’t so critical. You can wait a few hours for your answer, and if necessary, a few days. Conversely, other processing tasks are crucial, and the answers need to be delivered within seconds to be of value.

Real-time, near real-time, and batch processing

Type of data processing
When do you need it?

Real-time
When you need information processed immediately (such as at a bank ATM)

Near real-time
When speed is important, but you don’t need it immediately (such as producing operational intelligence)

Batch
When you can wait for days (or longer) for processing (Payroll is a good example.)

 

What is real-time processing and when do you need it?

Real-time processing requires a continual input, constant processing, and steady output of data.

A great example of real-time processing is data streaming, radar systems, customer service systems, and bank ATMs, where immediate processing is crucial to make the system work properly. Spark is a great tool to use for real-time processing.

Examples of real-time processing:

Data streaming
Radar systems
Customer service systems
Bank ATMs

What is near real-time processing and when do you need it?

This processing is when speed is important, but processing time in minutes is acceptable in lieu of seconds.

An example of this processing is the production of operational intelligence, which is a combination of data processing and Complete Event Processing (CEP). CEP involves combining data from multiple sources in order to detect patterns. It’s useful for identifying opportunities in the data sets (such as sales leads) as well as threats (detecting an intruder in the network).

Operational intelligence, or OI, should not be confused with Operational business intelligence, or OBI, which involves the analysis of historical and archived data for strategic and planning purposes. It is not necessary to process OBI in real time or near-real time.

Examples of near real-time processing:

Processing sensor data
IT systems monitoring
Financial transaction processing

What is batch processing and when do you need it?

Batch processing is even less time-sensitive than near real-time. In fact, batch processing jobs can take hours, or perhaps even days.

Batch processing involves three separate processes. First, data is collected, usually over a period of time. Second, the data is processed by a separate program. Thirdly, the data is output. Examples of data entered in for analysis can include operational data, historical and archived data, data from social media, service data, etc.

MapReduce is a useful tool for batch processing and analytics that doesn’t need to be real time or near real-time, because it is incredibly powerful.

Examples of uses for batch processing include payroll and billing activities, which usually occur on monthly cycles, and deep analytics that are not essential for fast intelligence necessary for immediate decision making.

Examples of batch processing:

Payroll
Billing
Orders from customers

Precisely can help keep your data fresh. Precisely Connect continually keeps data in your analytics platforms in sync with changes made on the mainframe, so the most current information is available in the data lake for analytics.

To learn more, read our whitepaper: Easy, Automated, Real-Time Data Sharing: What Can It Do For Your Business? 

The post The Difference Between Real-Time, Near Real-Time, and Batch Processing in Big Data appeared first on Precisely.

For the next three days, we have the COMMON Europe and COMMON (North America) Advisory Councils jointly meeting with us on the future of #IBMi – it’s an excellent opportunity! @COMMONug @CommonEurope #IBMPower

– Steve Will (@Steve_Will_IBMi)11:03 – Nov 14, 2022

IBM i in the cloud opens up a new world of possibilities for high availability (HA) and disaster recovery (DR). Real-time logical replication offers ultimate flexibility, providing a faster, lower-risk path for migration to the cloud, as well as for creating a switchable HA/DR cloud environment to support ongoing operations. Even for IBM shops committed to on-premise for their production servers, a cloud-based environment offers an on-demand HA/DR system that can be up and running in seconds, ensuring uninterrupted operations.

Cloud Environment Basics

IBM i in the cloud is different from a traditional on-premise deployment in several important ways. First and foremost, it involves virtual LPARs (i.e. virtual machines or VMs) configured via a cloud console and hosted by a cloud provider.

That has some important implications. LPARs will run on different hardware within the cloud environment. That means that serial numbers may change, or virtual serial numbers may be used. LPARs are, after all, mobile, so physical serial numbers may not be consistent from one day to the next. Live partition mobility events may occur, or hardware features may change.

IBM now supports virtual serial numbers, but because those are a relatively recent development, very few shops are using them currently. Your software licensing must be capable of handling scenarios in which your LPARs are shifted to a different hardware environment.

Whether you are running your IBM i workloads in the cloud or on-premise, the same underlying performance fundamentals still apply. Cloud LPARs must have adequate processing, memory, and storage resources to support your mainframe applications.

Tape Devices vs. Sync by Wire

IBM i in the cloud also means that access to physical tape drives or other hardware devices may be limited. Even if your cloud provider offers tape backup, shipping tapes for any restore operations will generally be impractical. This has important implications when it comes to disaster recovery.

LPARs are often scratch-loaded with an OS/TR/Cum level, and all OS maintenance is performed by the user. Full system restores may or may not be possible in an IBM i cloud environment, especially for large, complex environments. The recommended procedure, therefore, involves starting up the scratch-installed LPAR and syncing your environment over the network.

To get the ultimate infrastructure flexibility, choose a hardware-agnostic replication technology platform. Precisely Assure MIMIX, for example, has been serving customers in the IBM i space for years, offering replication between any hardware or cloud environment, whether that’s a mix of different IBM i server models, various IBM operating system versions, different storage types, physical or virtual servers, or different network types. That gives you complete flexibility to move from ground to cloud, cloud to ground, or cloud to cloud, supporting managed services or self-managed cloud environments.

IBM i High Availability and Disaster Recovery in the Cloud

Assure MIMIX can be used to migrate to the cloud, but once you’re there, Assure also makes it easy to design and implement a high availability and disaster recovery strategy for your entire IBM i landscape.

A common scenario involves an on-premise IBM i system, with HA/DR in the cloud. With real-time replication to that backup system, IBM shops can have a switchable HA/DR backup ready to run within mere seconds when it’s needed. Many customers prefer to run an HA system on-premise alongside their on-premise production server, with a DR system in the cloud. Assure MIMIX provides replication among the three systems to ensure that they remain in sync at all times.

Advanced scenarios involve cloud-to-cloud replication between two live production IBM i systems or targeted replication for backups.

Migration Challenges

When you’re migrating an IBM i system to the cloud, you’ll encounter some common challenges such as downtime, limited tape availability, testing for potential problems before cutting over to the new system, and fallback planning.

Traditional migration methods typically require some amount of downtime, which can be problematic for mission-critical operations. Typical migration scenarios involve between one and 48 hours of downtime. In a world that operates 24×7, that can be very problematic for most companies. By replicating your production system to a cloud environment using Assure MIMIX, you can begin working with a parallel production environment and cut over when ready.

This method also circumvents many of the problems associated with migrations that rely on tape backups. Tape is too slow and unreliable, and read errors can abort a migration midstream. In addition, many cloud environments lack tape availability altogether.

Regardless of the methods used to move data from on-premise systems to the cloud, it is essential to have a fallback plan in the event that the migration does not go as expected. With real-time replication technology in place, the fallback is essentially automatic. The production server continues to function normally until the cloud-based system is validated and approved for cutover.

Moreover, the newly migrated environment can be tested exhaustively, providing a very high level of confidence before the cutover goes into effect.

The cost of a failed migration can be high. A missed hardware return deadline, failed SLAs, or failure to meet regulatory compliance standards can all be costly events.

Migrate While Live, with Seamless HA and DR

Replicating your live system using Precisely’s technology enables migration with nearly zero downtime, ensuring that your data, applications, and critical objects are transferred to a secondary system accurately and reliably. Assure MIMIX is hardware-independent and OS-independent. In other words, IBM shops can transfer between two different OS versions, and to different hardware environments, including the cloud.

The same real-time replication technology can serve as a foundation for a long-term HA and DR strategy.  To learn more about HA and DR for your IBM i systems, read our free whitepaper, The Ultimate Buyer’s Guide to HA/DR Solutions.

The post Top Tips for Implementing IBM i HA in the Cloud appeared first on Precisely.

Establishing a foundation for IBM i cyber security

For many reasons, IBM i enterprises are overwhelmed with securing their IT infrastructure. 

You’re busy with the day-to-day maintenance, support and actual build of the IBM i applications that run your business. 

So, where does IBM i security fit in, what do you need to do and how do you manage it? 

Many IBM i shops already have systems, policies, procedures and tools already set-up that are working – and, despite a moving security landscape, many of these elements have been in place for years. 

If you’re considering reviewing, revising and replacing these systems, or you’re considering systems for the first time, join Andy Nicholson for a webinar on 30th November 2022 at 3pm (GMT) to discover a foundation for cyber security on IBM i.

 

//REGISTER for the webinar//

 

Surveys highlight IBM i cyber security as top concern

For the sixth year running, cyber security has ranked as the number one concern for IBM i IT professionals in HelpSystems (always) insightful IBM i Marketplace survey.

Why is cybersecurity at the top of the survey’s concerns? 

Well, Fresche, IBM and Forbes asked that same question within their 2022 cyber security report.  

The findings made grim reading.

43% of organisation don’t have any cyber security in place
61% of SMBs reported at least one cyber attack in 2021
A penetration testing study revealed cyber criminals can penetrate 93% of company networks
83% of organisation are not financially prepared to recover from a cyber attack
Nearly 50% of IBM i shops have little to no security knowledge and skills (with concerns that there are indeed limited security knowledge or skills within the IBM i community as a whole).

[SOURCE: Fresche, IBM & Forbes Cybersecurity in 2022 Report]

 

But the IBM i secure, right? Dispelling the myth that IBM i is impenetrable

IBM i is a safe platform, right? 

Well, not entirely.

For so many years, there’s been a general feeling that the IBM i is an impenetrable platform, immune to any kind of cyber threat.

And while the architecture of IBM i certainly makes it less likely than other platforms, unfortunately, there are always security fault lines in every business and flaws that can be exploited in every system.

And what about those particular fault lines and flaws? 

During the webinar, Andy will examine these  the top four risks as reported by IBM in greater detail – and crucially provide some practical guidance to combat these particular risks.

 

Malware

Malware or malicious software refers to worms, viruses, Trojans, and spyware — that provide unauthorised access or damage to the computer. 

Ransomware

Ransomware is a type of malware that locks down files, data or systems, and threatens to make public, erase, encrypt or even destroy data unless a ransom payment is made to the criminal.

Phishing

Phishing is a form of social engineering that tricks users into providing their credential or sensitive information. These are normally delivered via email or text messages disguised as a legitimate company – say a bank, government institution or business.

Inside Threats / Credential abuse

Typically, these come in the form of former employees, business partners, contractors or anyone who has had access to systems or networks. These can be considered a threat if they abuse their access permissions and as a result are invisible to firewall and intrusion detection systems.

 

Laying the foundation for security on your IBM i

Now we understand the types of cyber threats that can impact our IBM i, we can lay the foundations for protecting our infrastructure – and the application running on it.

During the webinar, Andy will provide practical advice and suggestions for IBM i security ranging from 

Profile management
User privileges
Managing passwords
Restricting access to your IBM i (closing the backdoor)
IFS security
Preventing phishing attacks through staff education
Backup and recovery 
High availability
Creating plans and policies
Importance of applying PTFs in a timely manner
Implementing field-level data encryption
Gaining cyber security accreditation
Having appropriate cyber security insurance.

 

Protecting your IBM i never stops

Cyber Security is not just ‘implement once’ and move forward, it’s a continually evolving landscape with new threats being introduced on a weekly, if not daily basis.  

Businesses need robust methods, policy and solutions in order to be able to counter and succeed.

Proximity has travelled this path, it’s not easy, but with support and some really terrific products on the market, it doesn’t have to be that difficult either. 

 

//Register for the webinar today to learn the foundations for your cyber security//

 

In the meantime, if you’d like to discuss your IBM i security, do not hesitate to drop Andy Nicholson an email on andy.nicholson[@]proximity.co.uk.

 

Interested in learning more about the IBM i products available to deliver IBM i (AS400 / iSeries) cyber security? Follow this link.

The post IBM i Security, it’s not easy but it doesn’t have to be hard either… appeared first on Proximity.